Absorption/desorption processes and systems for liquid immersion cooling

ABSTRACT

A two-phase liquid immersion cooling system is described in which heat generating computer components cause a dielectric fluid in its liquid phase to vaporize. Advantageously an absorption/desorption unit is employed having a carbon element and a controller configured to regulate the absorption unit.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority to PCT/US2019/060759 filed Nov. 11,2019 and PCT/US2019/051924 filed Sep. 19, 2019 which claims priority tothe following U.S. applications: Ser. No. 16/576,239 filed Sep. 19,2019, Ser. No. 16/576,309 filed Sep. 19, 2019, Ser. No. 16/576,191 filedSep. 19, 2019, Ser. No. 16/576,405 filed Sep. 19, 2019, Ser. No.16/576,285 filed Sep. 19, 2019, and Ser. No. 16/576,363 filed Sep. 19,2019, 62/897,457 filed Sep. 9, 2019, 62/875,222 filed Jul. 17, 2019,62/815,682 filed Mar. 8, 2019, Ser. No. 16/283,181 filed Feb. 22, 2019,62/768,633 filed Nov. 16, 2018, Ser. No. 16/165,594 filed Oct. 19, 2018,62/746,254 filed Oct. 16, 2019, and 62/733,430 filed Sep. 19, 2018. Allof the aforementioned applications are incorporated herein by reference.

FIELD OF THE INVENTION

The present inventions are directed to liquid immersion cooled computingsystems, namely liquid immersion cooled computing systems utilizingpressure and/or vapor management.

SUMMARY OF THE INVENTION

Traditional computing and/or server systems utilize air to cool thevarious components. Traditional liquid or water cooled computers utilizea flowing liquid to draw heat from computer components but avoid directcontact between the computer components and the liquid itself. Thedevelopment of electrically non-conductive and/or dielectric fluidenables the use of immersion cooling in which computer components andother electronics may be submerged in a dielectric or electricallynon-conductive liquid in order to draw heat directly from the componentinto the liquid. Immersion cooling can be used to reduce the totalenergy needed to cool computer components and may also reduce the amountof space and equipment necessary for adequate cooling.

In disclosed embodiments of the invention described below, the use ofvapor and pressure management systems, as well as power managementsystems may be utilized, individually or in combination, to createsignificantly improved computer systems utilizing liquid immersioncooling.

Embodiments of the disclosed inventions relate to a pressure controlledvessel which may be used to house a liquid immersion cooled computingsystem. In some embodiments, the pressure controlled vessel contains asufficient quantity of liquid dielectric fluid to substantially immerseheat generating computer components and also contains an atmospherecomprising gaseous dielectric fluid. Embodiments further comprise acondensing system in order to cool and convert gaseous dielectric fluidto liquid dielectric fluid. The disclosed pressure management systemallows the disclosed embodiment to operate under a vacuum, therebyreducing the temperature at which dielectric fluid vaporizes and thecomputing system operates. Disclosed embodiments allow for increaseddensity of computer components and/or computing power due to theimproved temperature management system described.

BRIEF DESCRIPTION OF THE FIGURES

FIGS. 1-2 show a schematic of a pressure controlled vessel according toan example embodiment.

FIG. 3 shows the exterior of an exemplary embodiment of a pressurecontrolled vessel 110.

FIG. 4 depicts an exemplary embodiment of a super structure containingmultiple pressure controlled vessels.

FIG. 5 depicts an exemplary data center embodiment showing multiplepressure controlled vessels connected to a central power supply.

FIG. 6 depicts an exemplary data center embodiment showing multiplepressure controlled vessels connected to each other in series.

FIGS. 7A-D depict an exemplary embodiment of a cooled computing systemwith an interior robotic arm, airlock, and exterior robotic arm.

FIGS. 8A-C show an example embodiment of a rack system.

FIGS. 9A-G show an example embodiment of a chassis for mounting variouscomponents.

FIGS. 10A-F show an example embodiment of a pressure controlled vessel.

FIG. 11 shows an example cooling and vapor management system for apressure controlled vessel.

FIGS. 12A-E show another embodiment of the vessel.

FIG. 13 shows an example of a self-contained vessel.

FIG. 14 shows an example of an outer housing for the self-containedvessel.

FIGS. 15A-D show an example magazine located on a platform capable ofextending out of the vessel.

FIG. 16 shows a vapor recovery system according to an exemplaryembodiment.

FIG. 17 shows an exemplary embodiment of a rack power distributionsystem.

FIG. 18 shows an example of a heating element for an immersion coolingsystem according to an example embodiment.

FIGS. 19A-B show a filter including three cores according to an exampleembodiment.

FIGS. 20A-B show an example robotic system.

FIGS. 21A-B show an example guide pin mechanism between a chassis and arack.

FIG. 22 shows example connectors with self-alignment features.

FIG. 23 shows an exemplary absorption unit.

FIG. 24 shows lowers aspects of an exemplary absorption unit.

FIG. 25 shows additional features of an exemplary absorption unit.

FIG. 26 shows upper aspects of an exemplary absorption unit.

FIG. 27 shows a cut-away of an exemplary absorption unit.

FIG. 28 shows representative connections pertaining to an exemplaryabsorption unit.

DETAILED DESCRIPTION OF THE INVENTION

In the following description, certain details are set forth such asspecific quantities, sizes, arrangements, configurations, components,etc., so as to provide a thorough understanding of the presentembodiments disclosed herein. However, it will be evident to those ofordinary skill in the art that the present disclosure may be practicedwithout such specific details. In many cases, details concerning suchconsiderations and the like have been omitted inasmuch as such detailsare not necessary to obtain a complete understanding of the presentdisclosure and are within the skills of persons of ordinary skill in therelevant art.

The equipment, components, systems, and subsystems of some disclosedembodiments below are described in terms of trade-names. It will beevident to those of ordinary skill in the art that the presentdisclosure may be practiced with many similar components whether or notsuch components are developed and/or sold under a particular trade nameand that the features and/or limitations associated with a particulartrade name components are not necessary to practice the disclosedinventions.

Dielectric Fluid

One aspect of immersion cooling is the use of a thermally conductive,but electrically substantially non-conductive or substantiallydielectric fluid. Examples of such fluids include some of the Novec™series of engineered fluids by 3M™ including Novec 7100, although thedescribed inventions are not limited to any particular dielectric fluid.Some immersion fluids typically have a boiling point at which it isdesirable to operate the cooled computer components. All computercomponents as well as other aspects of the disclosed systems arepreferably made of materials which are not soluble and do not otherwisebreakdown within the pressure controlled vessel when in contact with thedielectric fluid. In some embodiments, the boiling point of thedielectric fluid at standard atmospheric pressure may be less than about100° C., or less than about 80° C., or less than about 60° C., or lessthan about 50° C. or even lower. In some embodiments, the boiling pointof the dielectric fluid at standard atmospheric pressure is greater thanabout 60° C. or greater than about 40° C., or greater than about 30° C.or greater than about 20° C. Certain embodiments of immersion coolingfluids generally have a low vapor pressure. Some embodiments ofimmersion cooling fluids are fluorocarbons and/or fluorinated ketones.Certain embodiments of dielectric fluid may have a chemical formula of,or similar to, (CF3)2CFCF2OCH3, C4F9OCH3, or CF3CF2CF2CF2OCH3. Certaindielectric fluids comprise hydrofluoro ethers, methoxy-nonaflurobutane.

Other desirable characteristics of immersion cooling fluids include lowtoxicity, non-flammable, and/or low surface tension. In someembodiments, the immersion cooling fluid does not substantially harmcomputer components and/or the connections, wires, cables, seals and/oradhesives associated with computer components at the pressures andtemperatures utilized for liquid immersion cooling. Some dielectricfluids have a dielectric constant ranging from about 1.8 to about 8 anda dielectric strength of about 15 megavolts per meter (MV/m). In someembodiments, dielectric fluids have a dielectric strength of at leastabout 5 MV/m, or at least about 8 MV/m, or at least about 10 MV/m, or atleast about 12 MV/m. In some embodiments, dielectric fluids have adielectric strength of at most about 3 MV/m, or at most about 5 MV/m, orat most about 8 MV/m. In disclosed embodiments, any liquid in contactwith computer components 170 has a high enough dielectric strength toavoid damaging the computer components at the spacing and conditions ofthe specific application.

Some dielectric fluids have a critical heat flux of at least about 10W/cm2, or at least about 15 W/cm2, or at least about 18 W/cm2, or atleast about 20 W/cm2. Some dielectric fluids have a critical heat fluxof at most about 15 W/cm2, or at most about 10 W/cm2, or at most about 8W/cm2, or at most about 5 W/cm2.

FIG. 1 shows a schematic of a cooled computing system 110 according toan example embodiment. Embodiments of the disclosed cooled computingsystem 110 (or computing system, system, vessel, or pressure controlledvessel, all of which can be used interchangeably) may utilize a liquiddielectric fluid 140 to cool computer component 170 by immersing thecomponent into a bath of the fluid. As electricity is passed through thecomponent 170, the component 170 generates heat. As the component 170heats up, the performance of the component may be reduced or thecomponent may be damaged to the point of failure. It is advantageous tomaintain the various computing components at a stable and relatively lowtemperature. In some embodiments, computer component 170 may be kept atless than about 80° C., or less than about 70° C., or less than about65° C., or less than about 60° C., or less than about 55° C. In someembodiments, computer component 170 may be maintained at greater thanabout 60° C., or greater than about 50° C., or greater than about 40°C., or greater than about 35° C., or greater than about 30° C. As thecomputer component 170 heats up, heat is transferred to the liquiddielectric fluid 140 surrounding the component 170. When the liquiddielectric fluid reaches its boiling point, it will shift from a liquidphase into a gaseous phase and rise out of the liquid bath 142. Thecomponents 170 in the bath 142 of dielectric fluid may generally bemaintained at about the boiling point of the particular dielectric fluid140 being used.

When the liquid dielectric fluid is heated to the point of vaporizationat the pressure employed for a given application and becomes a gas,bubbles of the dielectric vapor will rise out of the liquid bath 142 andrise to the top of the system 110. The vapor is then cooled to be pointof condensing using condenser 130. Depending on the configuration of thesystem 110, the heating and cooling of dielectric fluid from liquidphase to vapor phase and back, can create a convection current as shownin FIG. 2.

In some embodiments, computer component 170 will be entirely submergedwithin liquid dielectric fluid 140 when the system is operating. Inother words, the upper portion of the computer component 170 is belowthe level of the dielectric liquid 140. It will be appreciated that asthe heat from computer components causes the dielectric fluid to changefrom liquid phase to gaseous phase, small bubbles of dielectric fluidvapor will be in contact with the computer components. Such componentswill still be considered entirely submerged within the liquid phase ofthe dielectric fluid. In some embodiments, the computer component 170may be submerged within the liquid phase of the dielectric fluid 140. Inone example embodiment, if any portion of a computer component,including but not limited to a motherboard, chip, server, card, blade,any portion of a GPU or CPU, and/or any peripheral component, is indirect contact with the liquid phase of the dielectric fluid 140, thecomputer component will be considered to be submerged. In certainembodiments, the computer component 170 may be at least partiallysubmerged within the liquid phase of the dielectric fluid 140. If thecomputer component 170 is not submerged, but is sufficiently cooled bydielectric vapor, the computer component will be considered to be atleast partially submerged.

In some existing immersion cooling systems, dielectric fluid must beconstantly added to the bath of dielectric fluid as the fluid isconsistently boiled off. Failure to add to the dielectric fluid to thebath 142 may result in the level of the dielectric fluid in the bath 142dropping until components are exposed to the gaseous atmosphere and notadequately cooled. This could result in decreased performance or damageto the component 170.

In some embodiments, there may be multiple operational modes which maybe accounted for with a fluid management system relating to thedielectric fluid in its liquid state. These modes may include, (1)Initial filling, which is the process by which dielectric fluid istransferred from a storage system into the vessel; (2) Continuousleveling, which is the process by which additional fluid is added, orexcess fluid is removed, to and from the vessel; (3) Unfilling, which isthe process by which the fluid is evacuated from the vessel and placedinto the storage system; and (4) Operational filtering, which is theprocess by which the fluid is continually cycled through a filteringsystem to ensure the removal of any particulates.

In some embodiments, the first three liquid management objectives, i.e.,initial filling, continuous leveling and unfilling, may be accomplishedthrough the same overall set of piping, pumps and valves. A dedicatedtank for storing liquid coolant may be used for the storage of new andexcess fluid which is removed and re-condensed during the vapormanagement process. A set of pipes and pumps may be used to bring thecoolant (or dielectric fluid) from the storage system to the vesselduring filling and leveling, and back out of the vessel and into thestorage system during unfilling operations.

In some embodiments, the fourth of the liquid management objectives, theoperational filtering, may be achieved through a series of skimmersand/or filters. The first stage may be a large particle filter locatedwithin the bottom of the vessel. The purpose of this filter is toprevent particles which are too large to be handled by the later stagesfrom entering the rest of the system. The second stage may be a mediumparticulate filter which sits in-line in the piping system between thefirst and third stage. This second stage medium particulate filter mayuse a small barrel style filter to remove particulates that were toosmall to be removed by the first stage filter but still too large to behandled by the third stage filter. The third stage filter may consist ofone or more parallel filters with support for various kinds of filterconfigurations. In some embodiments, the particular style of filter willbe dictated by conducting an analysis of the fluid after it has beenexposed and operating with a set of hardware components located withinthe vessel environment. Differing hardware and/or components are likelyto produce differing types of particulates and chemicals which may needto be filtered to ensure the long term life and efficiency of thedielectric fluid.

Pressure Management

In general, immersion cooling fluid must be kept free of dust, water,and/or other contamination. As the computer components 170 are in directcontact with the immersion cooling fluid 140, minor contaminants canresult in short circuits or damage to the computer components.Additionally, water or water vapor that may contaminate the dielectricfluid can reduce the dielectric properties, including, but not limitedto the dielectric strength, of the fluid as it becomes contaminated. Ifthe dielectric strength of the dielectric fluid is reduced, the computercomponents may short circuit or be otherwise damaged while in operation.One manner of reducing contamination is to operate an immersion coolingsystem in an enclosure which is kept at slightly higher or higher thanatmospheric pressure.

As the computer components 170 operate, the heat generated from theinitial use of the computer components causes some dielectric liquid 140to vaporize into a gas. If the immersion cooling system is confinedwithin a substantially enclosed housing, this vaporization typicallyincreases the pressure of the atmosphere within the housing. Pressurerelief valves, expanding enclosures, and/or other techniques may be usedto limit the increasing pressure and/or maintain the pressure within thehousing at or only slightly above atmospheric pressure. Maintaining aslight positive pressure in the enclosure may help to reduce theinfiltration of dust, water vapor, or other contaminants into theimmersion cooling computing system.

Current embodiments utilize an enclosed pressure controlled vessel 110(or cooled computing system 110) enclosure to contain the computingcomponent 170 and immersion cooling equipment, as well as the associatedpower supplies, networking connects, wiring connections, and the likewithin a pressure controlled vessel. In contrast to existing models, thepressure controlled vessel 110 may be maintained at least at a slightvacuum, thereby reducing the boiling point of the dielectric fluid 140to a temperature below its boiling point at standard atmosphericpressure.

By operating the computing and immersion cooling system under a vacuum,the components 170 may be maintained at the reduced, low-pressureboiling point of the dielectric fluid 140. This has the benefit ofincreased cooling which allows for more electricity to be passed throughthe various components 170 resulting in greater performance of thecomponents. By controlling the pressure in the pressure controlledvessel 110, the boiling point of the dielectric fluid 140 may also becontrolled, thereby allowing the same fluid 140 to be used in a broaderrange of conditions. Many embodiments benefit from cooler temperatures,however certain computer components 170 have an ideal range and sufferfrom reduced performance at temperatures below that range. Bycontrolling the pressure in the pressure controlled vessel 110, theboiling point of the immersion cooling fluid 140 may also be controlled.In certain embodiment, the disclosed pressure management system may beused to dynamically control the pressure, and thereby the boiling pointof the dielectric fluid 140 as the computing system is initiated, shutdown, or in response to other changing conditions.

In addition to reducing the boiling point of the dielectric fluid 140 byoperating in a pressure controlled vessel 110 at less than ambientpressure, a computer component 170 itself may be modified in order tomore efficiently transfer heat away from itself and into the dielectricfluid 140. By increasing the surface area of a component 170, forexample, a chip, which is exposed to the liquid dielectric fluid 140,heat transfer between the component 170 and the bath 142 of dielectricfluid 140 may be increased. An exemplary device for increasing surfacearea may be a copper boiler or a copper disc, which may be adhered to achip of other computer component 170. In certain embodiments, theadhesive used will be selected based on its ability to transfer heat andits solubility in the dielectric cooling fluid. Preferred adhesivesexhibit high thermal conductivity and low solubility in the selecteddielectric fluid.

FIG. 1 shows a schematic of an exemplary embodiment of the disclosedcomputing system. Embodiments of the disclosed systems include apressure controlled vessel 110 (or the cooled computing system 110), apressure controller 150, an immersion cooling system comprising at leasta volume of dielectric fluid 140 and a condensing structure 130, and thedesired computer components 170. A pressure system may be configured tomaintain the desired degree of reduced pressure. The pressure controlledvessel 110 may be configured to maintain a negative pressure while stillallowing multiple penetrations into the pressure controlled vessel 110for various connections including, but not limited to power, data,networking, cooling water, and/or communications systems. Someembodiments utilize hermetic and/or marine grade connections. Operatinga computer system within a pressure controlled vessel 110 at less thanambient pressure requires a series of modifications to the system as awhole. These modifications are discussed below and some are readilyapparent to one of ordinary skill in the art.

FIG. 3 shows the exterior of an exemplary embodiment of a pressurecontrolled vessel 110. In some embodiments, the disclosed pressurecontrolled vessel 110 is at least about 2 feet tall, or at least about 3feet tall, or at least about 4 feet tall, or at least about 5 feet tall.In some embodiments, the pressure controlled vessel is at most about 3feet tall, or at most about 4 feet tall, or at most about 5 feet tall.

In certain embodiments, the pressure controlled vessel has an interiorvolume of at least about 100 cubic feet, or at least about 150 cubicfeet, or at least about 200 cubic feet, or at least about 250 cubicfeet, or at least about 300 cubic feet, or at least about 350 cubicfeet, or at least about 400 cubic feet.

In some embodiments, the pressure controlled vessel will be configuredto contain about 12 vertical inches of liquid dielectric fluid and about36 vertical inches of dielectric fluid vapor while in operation. Incertain embodiments, the ratio of liquid volume to gaseous volume helpsto create a convective current and direct gaseous dielectric vaportowards condensing structures which turn the vapor back into a liquid.In some embodiments, the pressure controlled vessel is configured tocontain a ratio of a volume of liquid dielectric fluid to a volume ofgaseous dielectric fluid of about 1:6 during operation. In otherembodiments, the pressure controlled vessel is configured to contain aratio of a volume of liquid dielectric fluid to a volume of gaseousdielectric fluid of about 1:3, or about 1:5, or about 1:8 or about 1:10,or about 1:15 during operation.

In one example embodiment, the pressure management system may include apressure controller 150. The pressure controller 150 can be a source ofvacuum, e.g., the pressure controller 150 may be a vacuum pump which maybe connected to the pressure controlled vessel 110. In some embodiments,the vacuum pump 150 may be remote and the vacuum may be transmitted tothe pressure controlled vessel 110 using piping and/or tubing. Inpreferred embodiments, a pressure sensor 180 is contained within thepressure controlled vessel 110 and used to regulate and/or maintain thedesired negative pressure within the pressure controlled vessel 110. Insome embodiments, the pressure sensor 180 and/or a pressure regulator190 may be connected to a processor which monitors the pressure in thepressure controlled vessel 110 using the pressure sensor 180 andregulates the pressure using the pressure regulator 190.

Some embodiments comprise operator protection mechanisms. In one exampleembodiment, the operator protection mechanism may be a locking mechanismthat precludes the system from operating if any of the lids or servicepanels to the pressure controlled vessel are not in place. In oneexample embodiment, the operator protection mechanism may include acontroller to immediately power down the system in the event of anunauthorized breach of one of the doors or panels of the pressurecontrolled vessel. In addition to providing a life safety feature, theoperator protection mechanism may also provide an enhanced operationssecurity feature for deployments where sensitive data is housed withinthe vessel. By ensuring that the equipment cannot be accessed duringnormal operation without shutting down power to the system, a high levelof assurance can be achieved in the efficiency of disk protectionmechanisms. Furthermore, in some embodiments, the disk protectionmechanisms may use runtime stored encryption keys to protect data atrest on the pressure controlled vessel.

In certain embodiments, in addition to denying unsafe access to thepressure controlled vessel, sensors may be placed to ensure that thesystem is operating as designed. The primary sensor package may includetemperature sensors in the vapor space; temperature sensors in theliquid space; humidity sensors in the vapor space; and/or pressuresensors in the vapor space. These sensor readings may be monitored bysoftware and/or by human operators to ensure that the system isoperating in a safe and correct fashion. In some embodiments, the sensordata will be recorded or later analyzing.

In some embodiments, additional sensors may be incorporated within thevessel or the super structure (defined below). Such sensors couldinclude, for example, FLIR based heat imaging cameras; VESDA or otherforms of aspirating smoke detectors; and/or refrigerant leak detectorsdesigned to detect a leak of the dielectric fluid into the surroundingenvironment.

In some embodiments, the vessel and/or super structure may be equippedwith indicator lights relating to the operational status of the system.

Although the cooled computing system 110 is sometimes referred to as thepressure controlled system 110, one or ordinary skill in the artrecognizes that many, if not all, of the benefits of the cooledcomputing system 110 can be realized without using a “pressurecontrolled system.”

Vapor Management System

Liquid immersion cooling systems may be operated in different ways. Somemay operate by continuously cooling the immersion fluid directly. Othersmay operate by allowing the liquid to reach its maximum liquid phasetemperature and then boil into a vapor phase. Immersion cooling systemswhich operate by allowing the liquid to evaporate are called two-phaseimmersion cooling systems. Two-phase immersion cooling systems oftenallow the dielectric fluid to boil and/or vaporize and regularly addadditional fluid to replace the fluid which is lost to the atmosphere.

Disclosed embodiments utilize a liquid immersion cooling system which iscontained within a pressure controlled vessel 110. This has theadvantage of not losing the dielectric fluid 140 even after it hasconverted to a gaseous form. In a closed, or substantially closedpressure controlled vessel 110, the gaseous dielectric fluid may becondensed and added back to the bath 142 of the liquid dielectric fluid140 which is actively used to cool the computing components 170. Thecondensing step may be performed in any convenient manner, for example,by running process water through a thermally conductive tube. Condensingstructures 130 may include radiator fins and/or similar equipment whichincreases the surface area of the condenser, thereby allowing greaterand/or more rapid condensation of the gaseous dielectric fluid andreturning it to a liquid form. In some embodiments the process water isat ambient temperature and is not actively cooled. In other embodiments,the process water may be chilled using evaporative cooling, dry coolingtowers, and/or other method of chilling process water known in the art.

In some embodiments, there may be two interfaces between a pressurecontrolled vessel and external systems. The first may be the processwater supply interface. This may be a pipe which delivers process waterfrom a facility which provides chilled process water to a distributionmanifold on the pressure controlled vessel. The second may be theprocess water return interface. This may be a pipe which returns theprocess water to the facility which provides chilled water. The processwater may be returned to the facility after the process water has flowedthrough the pressure controlled vessel and associated coolingcomponents. Cooling components may include, for example, condensers,condensing coils, and/or radiators within the vessel as well as coilswhich reject heat from the exhaust of any powered components including,for example, motors, pumps, and/or utility cabinets. In someembodiments, there may be two interfaces between a super structure andexternal systems. The interfaces may be similar or substantially similarto the two interfaces between the pressure controlled vessel andexternal systems.

In some embodiments, the location of the condensing structures 130within the pressure controlled vessel 110 may be configured in order tooptimize the flow of vapor phase dielectric fluid and increase the rateand/or efficiency of condensation. In some embodiments, the geometry ofthe pressure controlled vessel 110 itself may be controlled in order toincrease the rate and/or efficiency of condensation.

In one example embodiment, the location of the condensing structures 130may facilitate and optimize placement (e.g., by a robot) of the computercomponent 170 within the vessel (or removal of the computer component170 from the vessel). For example, the condensing structures 130 can beplaced on a side (or a sidewall) of the vessel such that the condensingstructures 130 are not situated in between a lid of the vessel and thecomputer component 170. As such, when the lid is opened, a robot candirectly remove the computer 170 without any interference with thecondensing structures 130. This arrangement of the condensing structurecan streamline placement and removal of the computer component 170,thereby can offer significant benefits in autonomous operation of thevessel. In one example embodiment, the condensing structures 130 can belocated above a shelf within the vessel.

As shown in FIGS. 1-3, in one exemplary embodiment, a pressurecontrolled vessel is about 10 feet long, about 4 feet wide, and about 4feet tall. A bath 142 may be created within the pressure controlledvessel 110 using about 130 gallons of Novec™ dielectric fluid 140. Thisleaves a layer of liquid dielectric fluid about 12 inches deep in animmersion cooling tank at the bottom of the pressure controlled vessel,while the majority of the pressure controlled vessel volume is gaseous.The ceiling of the pressure controlled vessel is lower in the middle ofthe structure running lengthwise. The ceiling and/or lid 120 anglesupward and raises as it approaches the sidewalls of the pressurecontrolled vessel 110. Condensing structures 130 run lengthwise on twosides of the pressure controlled vessel 110. The condensing structures130 in this exemplary embodiment may be about 12 inches wide and about24 inches tall and run substantially the entire length of the pressurecontrolled vessel 110. The condensing structures 130 include radiatorlike material with high surface area fins which are cooled using flowingprocess water. Some embodiments may additionally or alternativelycomprise a heat exchanger.

As shown in FIG. 2, the structural arrangement within the pressurecontrolled vessel 110 directs a convective flow of dielectric fluidvapor as it rises from the liquid bath 142 after boiling. The structuralarrangement directs the convective flow up towards the ceiling of thepressure controlled vessel where the flow is directed toward the highsurface-area of condensing structures 130 and condensed back into aliquid form. The dielectric fluid 140 then flows back into the liquidbath 142. In this manner, the total amount of dielectric fluid 140 maybe conserved within this closed housing. The use of convective currentto circulate dielectric fluid vapor allows disclosed embodiments tooperate in the absence of a mechanical pump for circulating thedielectric liquid, thereby reducing the total energy usage of thedisclosed system.

Certain embodiments may utilize additional tanks and/or storagecontainers of dielectric fluid which may be used during star-up and/orshut-down of the system, in the event the pressure controlled vesselmust be opened, and/or to allow redundant and robust control of thelevel of liquid dielectric fluid.

FIG. 11 shows an example cooling and vapor management system 600 for apressure controlled vessel 110. In this example embodiment, the coolingand vapor management system 600 can include a chilled process waterstorage 611, which runs through the cooling coil 132 to causecondensation of the dielectric fluid 140. After passing through thecooling coil 132, the process water can proceed to a process waterreturn storage 612. The cooling and vapor management system 600 can alsoinclude a tank for vapor storage 614 and a tank for dielectric fluidstorage 615. The tanks 614 and 615 can provide dielectric fluid or vaporwhen needed, e.g., during star-up and/or shut-down of the system. In oneexample embodiment, the tanks 614 and 615 can be coupled via acondensing structure 616. In case there is an excess supply of vapor inthe tank 614, the condensing structure 616 can remove the vapor and addit as dielectric fluid to the fluid storage tank 615.

In some embodiments, during operation, the pressure controlled vessel ismaintained at about 3 psi less than ambient atmospheric pressure whichhelps to reduce the boiling point of the dielectric fluid and therebyreduce the operating temperature of the computer chips and othercomponents. In some embodiments, the pressure controlled vessel 110 ismaintained at least at about 2 psi below ambient pressure or at leastabout 4 psi, or at least about 6 psi, or at least about 8 psi, or atleast about 10 psi below ambient pressure.

In some embodiments, it will be necessary to select components with somedegree of tolerance for pressure fluctuations. It would be preferable,to use components which can withstand a wide degree of pressures toallow for manipulation of the coolant boiling point, and as such thegeneral operating temperature of the overall system, by adjusting theoperating pressure of the system. Given the operating nature of thetwo-phase system, standard operating conditions for some embodimentswould see a variance of between ±4 PSIg. In certain conditions, such asduring a rapid startup or shutdown of the system, a difference of threeadditional PSIg may be experienced. In some embodiments, system leveladjustments can be made to better control these variables and keep themwithin a more controlled and defined range.

In certain embodiments, the computer components 170 are operated atleast at about 3% less than ambient pressure, or at least about 5%, orat least about 10%, or at least about 15%, or at least about 20%, or atleast about 25%, or at least about 30% less than ambient pressure.

In some embodiments, the pressure controlled vessel is maintained,during operation at less than about 750 torr, or at less than about 710torr, or less than about 650 torr, or less than about 600 torr, or lessthan about 550 torr, or less than about 500 torr, or less than about 450torr, or less than about 400 torr, or lower. In some embodiments, thepressure controlled vessel is maintained, during operation at greaterthan about 650 torr, or greater than about 600 torr, or greater thanabout 550 torr, or greater than about 500 torr, or greater than about450 torr, or greater than about 400 torr, or greater than about 300torr.

Some embodiments utilize a vapor scrubbing process and/or initialpurging process in order to control the gaseous atmosphere within apressure controlled vessel. This process removes a portion of thegaseous atmosphere from the pressure controlled vessel and removesundesirable portions of the atmosphere such as air and water vapor.These, and other non-desirable portions of the atmosphere may beseparated based on the temperature at which the vapor condenses into aliquid. Due to the specialized nature and boiling point of thedielectric fluid, many naturally occurring contaminants may be removedusing this method. Removing the non-readily condensable fluids helps tomaintain the purity of the dielectric fluid. A fluid will be consideredto be not readily condensable if the condensation point of the fluid isgreater than about 20° C. lower than the condensation point of thedielectric fluid at standard atmospheric pressure or if the condensationpoint of the fluid is less than 10° C. at standard atmospheric pressure.

During maintenance, startup and/or shut down operations, a blanket ofinert gas, such as nitrogen, gas may be introduced into the pressurecontrolled vessel in order to reduce the amount of dielectric fluid lostwhen the pressure controlled vessel is opened and/or exposed toatmospheric conditions. As shown in FIG. 11, the cooling and vapormanagement system 600 can include an inert gas tank 613, which cansupply inert gas to reduce dielectric fluid loss.

Some disclosed embodiments may include a substantially self-containedserver and/or computing system. In some embodiments, specialized sealsand/or connections may be utilized to reduce the total number ofpenetrations into the pressure controlled vessel 110. Some embodimentscombine power, water, vacuum, and networking connections into a bundleof lines in order to minimize the penetrations into the pressurecontrolled vessel in order to reduce the potential for leaks while thesystem is under vacuum.

FIG. 4 depicts an exemplary embodiment of a super structure containingmultiple pressure controlled vessels. In this example embodiment, twopressure controlled vessels 110 are pre-plumbed, pre-wired and housedwithin a modular super structure 210. This allows for embodiments to bepre-fabricated and delivered as substantially complete, self-containedsystems. The modular system may be configured to be connected to othermodular embodiments of the disclosed computing system. In someembodiments, the modular super structure 210 will require only a singlepower connection and will be pre-wired with the appropriate electronicsto supply the required voltages to the computer components and/or otherelectronic components.

FIG. 5 depicts an exemplary data center embodiment showing multiplepressure controlled vessels connected to a central power supply. FIG. 6depicts an exemplary data center embodiment showing multiple pressurecontrolled vessels connected to each other in series. In these exampleembodiments, the pressure controlled vessels 110 may or may not beplaced within a superstructure.

FIGS. 7A-D depict an exemplary embodiment of a cooled computing systemwith an interior robotic arm, airlock, and exterior robotic arm. In thisexample embodiment, an internal robotic arm 230 contained within thepressure controlled vessel 110 may be used to remove a component 170 anddeliver the removed component to an airlock 220. Using the airlock 220,the component 170 may be removed without substantially disturbing ordisrupting the pressure, atmosphere, dielectric fluid, and/or the otherconditions within the pressure controlled vessel 110. Once the component170 is removed from the pressure controlled vessel 110, a replacementcomponent may be introduced into the pressure controlled vessel 110using the airlock 220. The replacement component may then be installedby the internal robotic arm 230. The use of components which may beinstalled in a “slot-in” manner, such as a blade server and chassis, mayfacilitate this process significantly.

A disruption to a condition within the pressure controlled vessel may bedetected by a sensor, e.g., pressure sensor, placed within the pressurecontrolled vessel. The disruption may be indicated by at least a 10%deviation in that condition outside of the standard range of operatingconditions. A significant disruption to a condition within the pressurecontrolled vessel may be indicated by at least a 30% deviation in thatcondition outside of the standard range of operating conditions.

In certain embodiments, a self-contained diagnostic program may runwhich analyzes the performance of the components within the pressurecontrolled vessel 110. If a component 170 is not performing as desired,a robotic arm 230 may be used to remove and/or replace the componentautomatically. In this manner a self-healing, self-contained serverand/or computing system may be created. In certain embodiments, such aself-healing system may be pre-fabricated and pre-wired to create amodular unit which may be shipped or delivered to remote locations usingconventional methods to provide significant high-efficiency computingpower which requires limited set-up and/or maintenance.

In some embodiments, a first vapor management objective of cooling thevapor and causing it to condense from a gas state back to a liquid stateis achieved entirely within the closed system of the vessel through theuse of condensing coils. Process water will be piped through condensingcoils within the vessel. The shape and geometry of the vessel itselfwill encourage the flow of vapor from the bath area to the coil area andgravity will serve to pull the re-condensed liquid back into the batharea.

In some embodiments, a second vapor management objective of monitoringand maintaining the internal pressure of the vessel is achieved throughthe use of integrated pressure sensors within the vessel and use of apurge system. In some embodiments, the purge system will be used toremove excess vapor from the vessel and condense it back to a liquid forstorage in the liquid storage tank.

In some embodiments, a third vapor management objective of controllingand removing non-condensable components of the vapor which are presentduring system startup is accomplished via the same mechanism as thesecond objective. The purge system may be used to bring the system underpressure during its initial startup and to remove any non-condensablegases from the system.

In some embodiments, a fourth vapor management objective of controllingthe overlay of an inert gas may be accomplished using a dedicatednitrogen overlay feeding system. This overlay keeps the coolant belowthe top of the vessel, allowing for minimization of the loss of coolantduring periods where the vessel is opened to service the componentstherein. Dedicated piping from a set of nitrogen storage tanks through aset of dedicated overlay pipes within the vessel will allow for theadding of the inert overlay when the operator desires to open thesystem. This gas, along with any other non-condensables, may be removedduring the non-condensable removal process which may occur at systemstartup. The overall vapor management process may be managed andmonitored through the control system software based on user commands andsystem state monitoring.

Ballast Blocks

In some embodiments of the disclosed system, such as that shown in FIG.1, the pressure controlled vessel 110 may include a deeper bath portion142 for containing the majority of the dielectric fluid 140 and abroader shelf area 112 adjacent to the bath. The boards, cards, chips,blades, and/or any other computer components 170 are substantiallycontained within the deeper bath section 142 of the pressure controlledvessel 110. The broader shelf area 112 may also contain liquiddielectric fluid 140 and/or collect dielectric fluid 140 that isre-condensed into the liquid phrase from the vapor phase. In certainembodiments, the depth of the dielectric liquid in the pressurecontrolled vessel 110 may be increased utilizing ballast blocks 160.Ballast blocks 160 may be used to occupy undesired volume on the shelf,thereby displacing any dielectric fluid 140 that would be on the shelf112 and raising the level of liquid without requiring the addition ofadditional dielectric liquid 140. In some embodiments, the ballastblocks 160 include riser feet 161 which allow fluid to flow underneaththe ballast blocks 160 so that condensed liquid can continue to flowinto the deeper bath portion of the pressure controlled vessel withoutthe flow being hindered by the ballast blocks 160.

The ballast blocks 160 may be made of any material that does notinterfere with the operation of the disclosed immersion cooling system.The ballast blocks may be made of materials including, but not limitedto, metals, rubbers, silicone, and/or polymers. Preferred materials arenot substantially soluble in the dielectric fluid. The blocks must bedenser than the dielectric fluid but are not required to be solid. Inpreferred embodiments, the blocks will have a handle or cut out whichallows the block to be more easily handled and manipulated. Someembodiments of the ballast blocks 160 utilize interlocking top andbottom sections so that the blocks maybe stacked on top of each other ina secure manner. The interlocking top and bottom reduce the risk of ablock damaging any nearby component if it slides or is otherwisedisplaced from its intended position. In some embodiments, theinterlocking top includes recessed portions which align with feet and/orrisers on the bottom portion such that the lowest block does not preventfluid flow and blocks may be securely stacked on top of the lowest blockin order to occupy a significant volume, thereby allowing the level ofdielectric liquid to be raised without requiring a significant amount ofadditional dielectric liquid to be added.

In some embodiments, the ballast blocks 160 are configured to run theentire length of the pressure controlled vessel 110 and/or shelf 112. Inother embodiments, the ballast blocks 160 may be substantially any sizewhich allows for the block to be handled. In such embodiments, multiplemodular ballast blocks may be configured to displace as large or assmall of a volume as desired. In some embodiments, a single ballastblock has an outer dimensions of about 2 feet long or about 3 feet longor about 4 feet long or longer and about 6 inches wide, or about 8inches wide, or about 12 inches wide, or wider, and about 1 inch tall,or about 3 inches tall, or about 6 inches tall, or about 8 inches tallor taller.

The Super Structure

The disclosed computing system consists of various components, all ofwhich may be attached, directly or indirectly to a physical superstructure 210, as shown in FIG. 4. The super structure 210 allows forpre-wiring and pre-plumbing of any required electrical, sensor, control,power, fluid control, pressure control, and/or communication systems.This allows for faster and simplified deployment in the field andtesting at the factory prior to delivery to the customer.

The super structure 210 is typically fabricated from metal componentsand may be skid mounted or configured to be handled with a forklift,hoist, or crane. In some embodiments, the super structure 210 isconfigured to fit within a standard container in order to facilitateshipping. The super structure 210 and associated components may beconfigured to weigh less than about 58,000 lbs total and may be dividedinto smaller subcomponents in order to facilitate shipping withoutrequiring special equipment. In some embodiments, super structure 210and the associated components will weigh less than about 50,000 lbs, orless than about 40,000 lbs, or less than about 30,000 lbs, or less thanabout 20,000 lbs. In some embodiments, super structure 210 and theassociated components will weigh more than about 5,000 lbs, or more thanabout 10,000 lbs, or more than about 20,000 lbs, or more than about30,000 lbs. Embodiments of the super structure 210 may be any size andor shape. Many embodiments are sufficiently large to contain multiplepressure controlled vessels 110, server racks 310, and the associatedliquid immersion cooling equipment as well as the necessary equipmentfor managing power delivery and distribution and network connectivity.

The overall design of the super structure 210 can be adjusted toaccommodate the unique aspects of each deployment, includingcustomizations to the types and quantities of power and process waterinterconnects to meet the needs of existing facilities.

The control and management systems for all of the components within thedisclosed pressure controlled vessel may be included as part of thedisclosed computing system. A preferred embodiment of the disclosedsystem includes all of the required mechanical systems to maintain andoperate a two-phase liquid immersion cooling environment, including therequired pumps, valves, regulators, vapor management systems, pressuremanagement systems, and other associated components.

The super structure 210 may be an open frame design, or may include sidepanels and access doors. This allows for deployment within existingstructures or outside in field locations. The super structure 210 may bemodified to include weatherization features, allowing for deployment inharsh environments. In some embodiments, the super structure may be askid/module framework.

Various systems, features and/or capabilities may be incorporated intothe super structure 210 to support, monitor, and manage the othercomponents of the pressure controlled vessel and any environmentscontained within or associated with the pressure controlled vessel. Insome embodiments, such systems may include fire detection and/orsuppression capabilities, dedicated air condition and/or environmentalmanagement capabilities, security features such as access control,and/or surveillance features among many others.

The Power System

Some embodiments of the super structure 210 are designed to acceptvarious means of electrical inputs and connect them to an existing powerdistribution system built within the super structure. One of manyexemplary embodiments includes a 415V input to a main breaker, which isthen distributed to a series of power shelves which converts the 415V ACinput to 12V DC output. In preferred embodiments, this conversion occursin substantially one conversion step, thereby reducing the lostefficiency typically associated with such conversions. Traditionalcomputer server locations typically convert incoming industrialelectricity from a high AC voltage, such as 415V to a reduced AC voltagesuch as 120V. This conversion results in a loss of energy to heat. Undercommon circumstances, this may result in about a 6% loss of energy.Then, the 120V electricity must be further converted to DC current forthe use by various computer component. This second conversion results ina second, about 6% loss of energy to heat. By directly convertingindustrial electricity of about 415V to about 12V DC, the total loss ofenergy to heat can be reduced

Another exemplary implementation may include the connection of a 480V ACinput to a power shelf which converts the 480V AC input to 48V DCoutput, which is then distributed to a series of intermediary powersupplies which converts the 48V DC input to various DC outputs,including, for example, 12V, 5V, 3.5V 3.3V and others.

On some implementations, there may be a single set of power supplies, orthere may be multiple power supplies operating at different input andoutput voltages. The exact configuration will be adjusted to meet theneeds of the particular equipment being installed and depending on theconditions of the application. The particular design of a power systemmay be adjusted to meet the needs of the particular environment in whichthe disclosed computing system is being deployed. Customizations mayinclude the type, capacity, and interfaces for both input and output ofpower to the system.

In some embodiments, a rack power distribution system may comprise amodular power supply system and/or set of modular power supply systems.The specific configuration of the modular power system or systems is notparticularly critical so long as it can deliver the desired quantity andtype of power to the rack. Accordingly, modular power systems may beconfigured in parallel or in series or in a combination thereof toprovide one or two or even multiple power distribution pathways. Thespecific pathways to the rack may be direct or indirect and often maydepend upon the components involved, power quantity and type, and/ordesired configuration. If desired, the pathway to the rack may involvedistributing power to a chassis located within the rack. The powerdistributed may be delivered at one or more desirable voltages which mayvary depending upon the configuration and components. In some casesdesirable voltages may include, for example, 12V, 5V and/or 3.5V. Insome embodiments, if a chassis is employed, then it may employ one ormore subsystems. Such subsystems may include any desired subsystem thatdoes not interfere with the desired quantity and type of power to bedelivered to the rack. As but one example a power-on-package subsystemmay be employed. Such a package may accept AC current and convert to DCcurrent and/or vice versa depending on what is desired. For example, aparticularly useful power-on-package subsystem may be designed to acceptinput power at 208, 240, 380, 400, 415, 480, and/or 600 volts AC andconvert that power directly to DC power, e.g., 48V DC.

The modular power supply system or systems may be powered directly orindirectly in any convenient manner. For example, a modular power supplysystem may be powered directly via a primary electrical distributionsystem within the chassis. Depending on the type and quantity of powerand other components, a chassis could use an interface, such as a set ofspring loaded pins or other suitable connector interface to establishelectrical continuity between the power distribution pathway and thechassis itself. Continuity could then be established between thatinterface connector and any desired power supply input interface ondesired servers or other computing components located on the chassis. Insome embodiments, a power-on-package module may be utilized within eachchassis to convert the voltage to the appropriate levels directly at thechassis itself. This could be used for various types of powerdistribution but may be especially useful with, for example, 48Vdistribution. FIG. 17 shows an exemplary embodiment of a rack powerdistribution system 950. In this example embodiment, a rack 310 canreceive an AC input 960 at an AC interface 311 of the rack 310. Thepower distribution system 950 can generate a DC output 320 anddistribute the DC output 320 to one or more chassis 400.

In some embodiments, ensuring supply of reliable power to the computercomponents within a rack is a primary concern. To that end, someembodiments use blade level power supplies or computer component levelpower supplies which may supply a certain input voltage and provide therequired output voltage to the blade and/or component level powersupplies. Some embodiments incorporate multiple power supplies into eachblade to provide redundancy.

In some embodiments, one or more switches may require power. Anexemplary switch may be a standard datacenter grade switch with theappropriate interfaces to connect to the backplane and provide racklevel communications to each blade. Some embodiments distribute only asingle voltage, and this can be accomplished by a power rail andinterface system with connectors to serve as the interface between apower rail and each of the blades, delivering the voltage eitherdirectly to the power supply input rails or via an intermediaryconnector to sit in between the power supply power leads and a racklevel voltage distribution system.

In some embodiments, there may be one or more power rails whichdistribute the primary voltage along the bottom of the rack. This railmay be fed from one or more primary power rectifiers, likely locatedoutside of the pressure controlled vessel and delivered to each rack viaa cable or busbar system. The use of higher voltage, for example, 48Volts, at this level may reduce the required current carrying capacityof the distribution system and may effectively interface between thedistribution rail(s) and the load interface.

In some embodiments, there will be two primary power distributionsystems located within the super structure platform. The first is thePrimary Equipment Power System (PEPS) and the second is the SecondaryEquipment Power System (SEPS). The purpose of the PEPS is to provideelectrical service to the components within the vessel. This system maybe a high voltage, high current distribution system which accepts inputsvia either copper conductors or busbar systems and delivers it to theprimary power supplies responsible for providing operational current tothe chassis, computer components and/or other critical load equipment.The power will enter the super structure at a defined point and beterminated to a master service disconnect breaker. Upstream of thispoint will be the electrical service and all power redundancy componentsused in the system. This input will be at a high voltage, such as, forexample, 415 or 480 volts AC. The primary equipment load will be drivenby power supplies or rectifiers which are fed from a breaker paneldownstream of the master disconnect breaker.

The purpose of the SEPS is to provide electrical service to all of theinfrastructure support systems and components located within the superstructure. As the components required as part of the secondary equipmentinfrastructure may expect a lower input voltage, the SEPS may be poweredby a step-down transformer which is connected upstream of the PEPSmaster service disconnect breaker via a secondary service disconnect.

This arrangement will allow for the super structure support andinfrastructure systems, including all of those components which arepowered from the SEPS, to be turned on and operate even without primarypower being delivered to the remainder of the system components. Allaspects of the management and control systems, as well as the vaporcontrol system, may be able to operate independently of the operation ofthe PEPS.

In some embodiments, an uninterruptable power supply (UPS) is includedas part of or in addition to the power distribution system. Theincorporation of the UPS allows for continued operation of the disclosedcomputing system in the event of a temporary interruption to theexternal power supply.

Components of the disclosed power distribution system may include, butis not limited to, commercially available components such as, forexample, uninterruptible power supplies, DC power systems, AC powersystems, and/or power control and monitoring systems. Some suchcomponents may include, but are not limited to Vertiv products such as,for example, Liebert and/or Chloride UPS products, dual conversiononline UPS, line-interactive UPS, stand-by UPS, lithium-ion battery UPSand combinations thereof. The UPS products may be single phase or threephase. Other exemplary power distribution system components may include,for example, Emerson Network Power products, NetSure DC power systems,Vertiv, Liebert, Chloride, and/or NetSure power distribution units andrelated components, such as, for example, inverters, rectifiers,transfer switches, and combinations thereof. Commercially availablemonitoring units, controller units, and/or software related to suchcomponents may also be incorporated into certain disclosed embodiments.

The Pressure Controlled Vessel and Pressure Management Systems

Embodiments of the disclosed system include a pressure controlled vesselwhich is designed to house a two-phase liquid immersion cooling system.The pressure controlled vessel 110 contains a bath 142 of dielectriccooling fluid 140, condenser 130 with cooling coils 132 to condensegaseous phase dielectric fluid into a liquid, and the physicalmechanisms and/or equipment necessary to hold computer components 170and distribute power from the power system to the equipment andcomponents located within the pressure controlled vessel 110.

During operation, the pressure controlled vessel 110 may be kept at aslight vacuum. It will be appreciated that a variety of specializedconnections and considerations must be made in order to operate acomputing system within a pressure controlled vessel 110 which ismaintained at a negative pressure.

Some embodiments of the disclosed system use a series of fiber opticMedia Transfer Protocol (MTP) interfaces allowing connectivity of fiberinto the pressure controlled vessel 110 in addition to break out panelsand cable trays to distribute the fiber to the racks 310. Thisarrangement reduces the total number of penetrations into the pressurecontrolled vessel 110 reducing the likelihood of leaks in the vessel.

Some embodiments of the pressure controlled vessel 110 include sensorsto ensure safe operation. These sensors may include, but are not limitedto, temperature sensors, fluid level sensors, pressure sensors 180,gaseous partial pressure sensors, position sensors, electrical sensors,microphones, and/or cameras to ensure and/or automate operations of thesystem.

In one example embodiment, temperature sensors may include, but notlimited to, sensors for measuring the temperature of the gaseous phasewithin the pressure controlled vessel 110, sensors for measuring thetemperature of the liquid phase within the pressure controlled vessel,sensors for measuring the temperature water and/or other process fluids,and/or sensors for measuring the temperature of the other componentsincluding the computer components 170. In some embodiments,thermocouples, thermistors, and/or silicone sensors may be utilized tomeasure the temperature of computer components. In some embodiments, thesystem may rely on information provided by the components themselves andretrieved or monitored through the use of a generally acceptedcommunications protocol, such as a device provided API or otherprogrammatic interface, such JSON via HTTPT or SNMP, to determine theequipment temperature.

Some embodiments may include various life safety features to ensure thesafety of users. These features may include, but are not limited to,automatic electromagnetic locking mechanisms, fail safe systems, fireand/or smoke detection and/or suppression systems, ventilation systems,and/or back up lighting. In certain embodiments, these features may beincorporated as part of a comprehensive platform.

Certain embodiments include an automatic vapor detection based leakdetection system to ensure that any loss of fluid in the pressurecontrolled vessel is rapidly detected. These systems may includepressure sensors 180 within the pressure controlled vessel 110 whichmonitors the pressure in order to ensure there are no substantial leaksand/or gas sensors positioned on the exterior of the pressure controlledvessel which detect the presence of any dielectric vapor which may haveleaked out of the pressure controlled vessel.

The particular design, arrangement, and/or layout of an embodiment ofthe disclosed system may be adjusted based on the conditions in which itis deployed. In some embodiments, the size, material, internal systems,component mounting and configuration options, interfaces between thepressure controlled vessel 110, the computer components 170, and powersystems may all be adjusted based on the conditions in which the systemis utilized.

The Rack System

FIGS. 8A-C show an example embodiment of a rack system 310 (or rack310). The rack 310 may serve as an intermediary between the electricaland communication systems installed within a pressure controlled vessel110 and the computing equipment 170 to be installed within the rack 310.Computer components 170 can be mounted on the rack 310 in order tocontrol the spacing, orientation, position, and/or configuration of thecomputer components 170 in the pressure controlled vessel 110. In oneexample embodiment, each computer component 170 may be installed in achassis 400 before being mounted on the pressure controlled vessel 110.

The rack 310 may be any physical structure which may be used to mountcomputer components 170 including but not limited to any frame, bracket,support, or other structure. Computer components 170 will be consideredmounted to the rack 310 when they are attached, directly or indirectly,to the rack 310 and held in a substantially stationary position. Someembodiments may include the use of dedicated mechanical guide plates asmounting mechanisms, wiring harnesses attached to bulkhead fittings,and/or through the use of intermediary power supplies and a backplanereceiver 331 to distribute power and signal within the rack.

The particular design of the rack system 310 may be adjusted based uponconditions under which the system is deployed. Some embodiments of therack 310 may include a dedicated switch. In some embodiments, the uplinkinterfaces may be connected via fiber infrastructure and/or the downlinkaccess interfaces may be connected to computing equipment 170 within therack via the backplane receiver 331 interface or any other suitable wayof connecting computing equipment.

In certain embodiments, the rack system 310 may include housing for oneor more intermediary power supplies which may distribute the appropriatevoltages from a power interface to other equipment installed within therack 310. The interfaces to interconnect the power from the distributionsystem to the intermediary power supplies may be incorporated into thedesign of the rack 310 to allow it to be removed and/or replaced with analternative rack configuration by disconnecting the interfaces betweenthe various rack, power, and communication systems.

FIG. 8A shows a top view of the rack 310. In this example embodiment,the rack 310 includes an AC interface 311 and a data interface 312. Therack 310 also includes a pair of power supplies, a power supply 313 anda redundant power supply 314 (or backup power supply). The rack 310 mayalso include rectifiers and a controller. The redundant power supply 314(and/or the rectifiers and controller allow the rack 310 to be quicklyrepaired or even to continue functioning if the primary power supplystops functioning. The rack 310 may optionally include a converter 315.The rack 310 is configured to receive a plurality of chassis 400 andhold the chassis 400 in a substantially stationary position.

In some embodiments, the entire rack 310 may be submerged in dielectricfluid. This may include submerging the rectifier, power connections,and/or data connections in dielectric liquid during operation. In orderto reduce and/or eliminate plastic contamination of the dielectricfluid, in some embodiments, plastic insulation and/or cable shieldingmay be eliminated. In such embodiments, the dielectric fluid may serveto insulate the otherwise exposed cables and/or connections.

FIG. 8B shows a perspective view of the rack 310 including a pluralityof chassis 400. The disclosed configuration of the rack facilitates thehot swappability of the chassis 400. In this example embodiment, therack 310 can include a plurality of AC cables 318 which connect the ACinterface 311 to the power supply 313 and/or the redundant power supply314. The power supply 313 and/or the redundant power supply 314 cangenerate a DC output 320 which can be delivered to the backplanereceiver 331 via a DC cable 321. The rack 310 can also include aplurality of data cables 319 which connect the data interface 312 to thebackplane receiver 331. The backplane receiver 331 may be used to supplydata from the data connections on the bottom of the chassis 400 to adata connection at the top of the rack.

FIG. 8C shows a side view of the rack 310. In some embodiments, theracks 310 provides mechanical stabilization and/or housing for thechassis 400 and its components. Additionally, the racks 310 facilitatethe routing of power and data cables from the top of the racks 310,where they are generally accessible within the vessel, to the bottom ofthe racks 310 where they connect with the chassis 400.

The Chassis and Interface Systems

In one example embodiment, the purpose of the disclosed chassis system400 is to serve as a standardized physical intermediary componentbetween traditional and/or purpose built computing components 170 andthe disclosed rack system 310. In one example embodiment, the purpose ofthe backplane receiver 331 is to provide a slot-in interface between thechassis 400 and the rack 310, allowing for the distribution of power andsignals between the power supplies in the power system and the networkswitches in the communication system with the various computingcomponents 170 installed within the chassis 400.

In some embodiments, the pressure controlled vessel of the presentdisclosure can include at least one rack 310, which can include one ormore servers, e.g., blade servers. Each server can be attached to achassis 400 (also called server case or case). FIGS. 9A-G show anexample embodiment of a chassis 400 for mounting various components 170.The chassis can facilitate installation of the servers on the racks ofthe pressure controlled vessel or their removal from the system. In someembodiments, other electronic components of the pressure controlledvessel can be mounted in a chassis. For example, computer components orhardware such as a motherboard, chip, card, any portion of a GPU or CPUcan be installed in a chassis. As another example, components such as apower supply, a power interface, or a network communication interfacecan be mounted in the chassis.

In one example embodiment, the chassis can serve as a common interfacebetween a component (e.g., server) and the pressure controlled vessel.The chassis can provide a variety of mounting, power, and connectivityfeatures which can be customized based on the nature or design of thecomponent. In other words, various aspects of the chassis can bemodified based on the design specification of the component. As such,the chassis can accommodate almost any model or type of hardware. Forexample, the chassis can facilitate usage of specifically designedhardware or off-the-shelf hardware.

Embodiments of the chassis 400 may include components designed to allowfor the adaptation of existing commercial components, the use of customdesigned components, and/or the use of specialized chassis forparticular applications. Embodiments may include adaption kits forstandard motherboards, and specialty components. In particularembodiments, such components include Gigabyte motherboards with NVidiaGPUs and/or Supermicro motherboards with Intel CPUs.

FIG. 9A shows a chassis 400 for mounting a server on a rack according toan example embodiment. In this example embodiment, the chassis 400 canbe a rectangular box including a back wall 410 and two sidewalls 420.The back wall 410 can include a plurality of holes 411 to facilitatecirculation of fluid within the chassis 400. The chassis 400 can includea guide rail 421 on each sidewall 420.

FIG. 9B shows a plurality of components inside of the chassis 400according to an example embodiment. In this example embodiment, the backwall 410 is removed. As such, FIG. 9B shows a server 430 including apower supply module 431, a GPU module 432, a CPU module 433 and aninterface card 434. In one example embodiment, the components inside thechassis 400 can comprise the components used in a blade server, e.g., aCPU module 433 and a GPU module 432. Additionally, the components insidethe chassis 400 can include other components which are traditionally notincluded in a server, e.g., power supply module 431 or interface card434. Because the chassis 400 does not need the traditional air coolingequipment, the chassis 400 does not include any fans or heat sinks inthe chassis. As such, the chassis has a very thin profile relative tothe computing power of the chassis.

FIG. 9C shows a schematic drawing of the components within a chassis. Inthis exemplary embodiment, a server motherboard 445, a plurality ofpower supply modules 431, and an interface card 434 are mounted on achassis 400. Storage devices and/or other peripheral components may alsobe mounted to the chassis 400 along with a backplane interface 330and/or a power module and communication system module.

In one example, mounting interfaces can be added to or removed from thechassis so that a piece of hardware is fixed to the chassis. On aninternal surface of the chassis 400, provisions can be made which allowfor components (e.g., motherboard, GPU, CPU, interface card and otherrelated components) to be mounted to the chassis. These provisions arethe mounting interfaces. The specific arrangement of the chassis system400 may depend on the equipment and/or components that will be attachedto the chassis 400 and/or rack. Some embodiments of the chassis 400 mayfeature an interchangeable mounting plate that can be used for equipmentattachments. A set of standard attachment plates may be used for commonor frequently used components.

The styles and form factors of power and network interface moduleswithin the chassis system 400 may be adjusted based on the demands andrequirements of certain components and/or user specified equipment. Inone example, a power subsystem of the chassis can be modified to addressthe needs of a particular component. In another example, the size of thechassis can be designed to accommodate a piece of hardware of any size.In yet another example, the chassis can offer different networkingoptions depending the network connection card installed in the chassis.Because of these features and other features of the chassis, the chassiscan accommodate a variety of components. As a result, assembly orremoval of these components of the pressure controlled vessel can becomesimplified, and thus, automated. For example, a chassis can include ablade server and a robot can easily install or remove the chassis from arack of the pressure controlled vessel. As such, the robot can removeand replace a blade server without any human interaction, therebyminimizing human exposure to the dielectric fluid.

In one example embodiment, the chassis can include a microcontrollerwhich can be in communication with a management system of the pressurecontrolled vessel. The microcontroller can receive sensor data fromvarious sensors placed in or outside of the chassis. For example, thechassis can include a sensor for detecting whether the chassis isproperly placed in a rack. A server is properly placed in a rack if theserver can make a connection with the rack. The sensor can determinewhether the chassis is properly placed in a rack. As such, the sensorcan transmit data to the microcontroller, and using the data, themicrocontroller can provide a signal to the management system indicatingwhether or not the chassis is properly placed in the rack.

In one embodiment, the microcontroller can be coupled to a switch whichcan power on or off the component mounted in the chassis. Themicrocontroller can receive a power on or off signal from the managementsystem, and in response to receiving the signal, the microcontroller cantransmit a signal to the switch to power on or off the component, e.g.,server. In one example embodiment, the microcontroller can receiveoperational data from the server and the microcontroller can relay thisdata to the management system. Operational data are key performanceindicators of a server and can indicate its performance. Operationaldata can include the speed of the compute, degradation in the compute,power consumption, temperature of the circuits and bandwidth of thesystem.

In one example embodiment, the microcontroller can monitor, manage andcontrol the electrical and communications facilities of a blade server.For example, indicators such as electrical current (i.e., amperage) andvoltage are monitored to make sure that the system can protect itself,e.g., there is no provision of over-current or under-current.

In one example embodiment, the chassis can include a structure which canenable a robot to grab and remove the chassis. For example, the chassiscan be the shape of a rectangular box having front, back and sidewalls.The chassis can also include a top wall and a bottom wall. The top wallof the chassis can include a plate which can make a coupling with arobotic arm. Using the plate, the robotic arm can grab the plate forunloading and other handling operations.

In one example embodiment, the chassis can include mechanical guiderails and positioning pins to ensure proper alignment and insertion ofthe chassis in a rack. Mechanical guide rails may be placed on thesidewalls of the chassis.

In one example embodiment, the chassis can include various features topromote fluid flow. For example, the chassis can be the shape of arectangular box having front, back and sidewalls. The chassis can alsoinclude a top wall and a bottom wall. In this example, at least one ofthe walls of the chassis can include fluid flow holes throughout thewall. For example, the back wall can include a plurality of holes whichcan facilitate fluid flow in and out of the chassis when the chassis isimmersed in a liquid bath.

In one example embodiment, a chassis can include apertures to ensurethat all fluid within the chassis drains when the chassis is removedfrom a liquid bath. For example, a rack can be located in a liquid bathto cool the computer components held by the rack. In order to remove aserver, a robot can grab the plate of a chassis and lift the chassis outof the rack (thereby removing the chassis from the liquid bath). Whenthe chassis is removed from the liquid bath, certain amount of fluid canremain within the chassis. The chassis can include notches or drains atthe bottom wall of the chassis to ensure that fluid can escape even ifthe pressure controlled vessel is not perfectly level. The notches ordrains can be at the corners of the bottom wall.

In one example embodiment, the chassis can include a power interfaceand/or communication interface. The interfaces can electrically couple acomponent mounted within the chassis to the rack and/or the pressurecontrolled vessel. The power interface and/or communication interfacecan be placed at the backplane. For example, a server mounted within thechassis can be connected, via various wires and cables, to the interfaceof the chassis. When the chassis is placed within the rack, theinterface can be electrically coupled to another interface connected tothe rack (i.e., backplane receiver) and/or the pressure controlledvessel. The electrical coupling between the two interfaces (e.g., thebackplane and the backplane receiver) can provide power to the serverand connect the server to a communication network within or outside ofthe pressure controlled vessel. The coupling between the two interfacescan be made automatically during the mechanical insertion of the chassisin the rack. Similarly, removal of the chassis from the rack candisconnect the server from rack and/or the pressure controlled vessel.

In some embodiments, providing standardized interconnectivity throughthe backplane interface 330 and communication system interfaces mayminimize the possibility for misconnection of data interfaces and reducethe need for connectivity troubleshooting.

In certain embodiments, the chassis 400 will include a set of standardpower and network interfaces. The network interfaces may be in the formof Cat6A or Cat7 compatible RJ45 interfaces for connection to 1G or 10GEthernet interfaces on equipment motherboards. In such embodiments, thepower interface may include a set of standard Molex style connectors forthe connection of standard motherboards and/or peripheral components.

In one example embodiment, the pressure controlled vessel can include aninternal database for storing information about the components installedwithin the system. The internal database can be a repository ofcomponents installed on the pressure controlled vessel. For example, theinternal database can store the make and model of every server and powersupply installed within the system. As the components of the system areexchanged or replaced, e.g., by a robot, the management system can keeptrack of the changes and update the information stored on the internaldatabase. The pressure controlled vessel can also be connected to anexternal database via a network.

In one example embodiment, each chassis can be associated with a uniqueserial number, e.g., displayed as a barcode on the chassis. When acomponent is placed within a chassis, the specification of the component(or the component's make and model) can be stored in the externaldatabase in association with the unique serial number. Subsequently,when the chassis is installed in the pressure controlled vessel, thepressure controlled vessel can look up the component by searching theexternal database for the unique serial number. For example, a roboticarm can scan the barcode on the chassis and the management system cansearch the external database using the barcode. The management systemcan update the internal database using the information obtained from theexternal database. Similarly, when a chassis is removed from thepressure controlled vessel, the robotic arm can scan the barcodeassociated with the chassis, and the management system can update theinternal database to indicate that the component mounted in the chassisis not installed in the system anymore.

In one example embodiment, a chassis can include an RFID tag. Therobotic arm of the pressure controlled vessel can include a scannerwhich can emit radio frequency to detect the RFID tag. When the roboticarm is handling a chassis, the robotic arm can scan the RDIF tag andprovide the unique serial number to the management system to update theinternal database.

In one example embodiment, the chassis can include an identificationplate which can contain a user specified asset identification number.This asset identification number can be associated stored in associationwith the component mounted within the chassis. In some embodiments, theidentification plate can be a chip configured to store the assetidentification number.

In one example embodiment, a chassis can include a pump to enhance flowof fluid within the chassis. In order to maximize heat exchange betweena component within a chassis and the liquid bath, the chassis caninclude a pump which can circulate the fluid within the chassis andaround the component. The pump can draw the fluid from various conduitsspread around the chassis and propel the fluid outside of the chassis orvice versa.

In one example embodiment, a chassis can include various conduits aroundthe chassis for drying the chassis and the component mounted therein.When a chassis is pulled from the liquid bath, some amount of liquid mayremain within the chassis or components therein. The chassis can includevarious conduits which can guide a flow of gas within the chassis oraround the component to facilitate drying of the chassis and components.In one example embodiment, the pressure controlled vessel can expose thechassis to a flow of gas before delivering the chassis to a user. Forexample, the chassis can include an input pipe for receiving the flow ofgas and the pressure controlled vessel provide the flow of gas throughthe input pipe.

FIG. 9D shows a bottom wall 415 of the chassis 400 according to anexample embodiment. In this example embodiment, the bottom wall 415 caninclude a power interface 416 and a communication interface 417. FIG. 9Dalso shows the guide rails 421 on sidewalls 420 of the chassis 400.

FIG. 9E shows a top wall 425 of the chassis 400 according to an exampleembodiment. In this example embodiment, the top wall 425 can include aplate 426 and a pair of handles 427. A robotic arm can pick up thechassis 400 using the plate 426.

FIG. 9F shows a sidewall 420 of the chassis 400 according to an exampleembodiment. In this example embodiment, the sidewall 420 can include aguide rail 421. FIG. 9F also shows the back wall 410, the handles 427,and the power interface 416.

FIG. 9G shows an exploded view of a bottom drain hole 450 of the chassis400 according to an example embodiment. In this example embodiment, thebottom drain holes 450 can be placed on the corner of the bottom wall415, sidewall 420, and the back wall 410.

FIGS. 10A-F show an example embodiment of a pressure controlled vessel500. In particular, FIG. 10A shows an exemplary embodiment of a vessel500, e.g., a 600 KW skid. The exemplary embodiment includes a modularskid. The vessel 500 may include a plurality of forklift tubes 514,which facilitate movement and transfer of the vessel 500 to a desiredlocation. The vessel 500 may receive a power and communication input 511and process water from process water pipes 512 with minimal penetrationsthrough the vessel itself. These connections may be positioned on thetop of the vessel in order to facilitate close packing of the modularvessel in a data center. In some embodiments, the connections may beplaced on the front and/or side of the vessel in order to accommodatevertical stacking of multiple modular vessels within a data center. Insome embodiments, a vessel may comprise a vertical spacer to facilitatevessels being stacked vertically on top of each other. The verticalspace may create additional space for connections, air flow, and/orinsulation between vessels. By vertically stacking vessels,extraordinary power density may be achieved on a square foot basis. Insome embodiments, the vessel 500 may include a power and communicationbox configured to receive the input 511 and distribute power and networkconnectivity throughout the vessel 500. The vessel 500 may include asealing lid 515, which may facilitate addition and/or removal ofcomponents from the vessel 500.

FIG. 10B shows another view of the vessel 500. In some embodiments, aninventory of replacement components may be stored within the vessel 500so that components may be replaced using a robotic system within thevessel without opening the vessel. The robotic system may be operatedusing the gantry motors 516. In such embodiments, when a componentbreaks or needs repair, a replacement component is installed into thesystem and the broken or otherwise removed component may be stored in acassette until the cassette is full. At that point, the cassettecontaining removed components may be removed from the vessel and a freshcassette with new replacement components may be inserted into the vesselfor future use. In some embodiments, the disclosed vessel is about 15feet long, about 7 feet wide, and about 10 feet high. In someembodiments, the disclosed system may provide for 600 KW of computingpower to be achieved in about 150 square feet.

In some embodiments, the vessel 500 may also include one or more bellowstank 517. The bellows tank 517 may be used to regulate pressure withinthe vessel. When the disclosed computing and/or cooling system isinitially activated, the expanding dielectric fluid may be directed tothe bellows tank so that it is not lost to the environment and/or toavoid pressure building up within the vessel. In some embodiments, thebellows tank 517 may be large enough to hold about two times the liquiddielectric fluid within the vessel.

FIG. 10C shows a section view of the vessel 500. The lower portion ofthe vessel 500 may contain a rack 310 and/or chassis 400 which containcomputing components. Above the rack is a condenser coil 132 which coolsand condenses any dielectric vapor. Power may be distributed within thevessel using a power bus bar 518. This allows power to be distributed tothe individual computing components in a hot-swappable manner. The powerbus bar 518 allows the vessel to receive external power using only oneor a small number of penetrations through the vessel. This designsimplifies installation and operation of the vessel system. In someembodiments, each power bus bar may serve 600 amps for the powersupplies to five racks. In such embodiments, there may be two sets ofbus bars, one on each side of the vessel. In some embodiments, the busbars do not include plastic insulation. Plastic may be regarded as acontaminant of some dielectric fluids and may be generally avoided insome embodiments.

In some embodiments, the vessel 500 may include a desiccant 519. In someembodiments, dielectric vapor may be removed from the head space of thevessel 500 and condensed in a manner that allows any non-condensableconstituents to be removed from the dielectric fluid. Water will notcondense under the same conditions as many dielectric fluids. As such,this system may be used to remove water contamination from thedielectric fluid.

In some embodiments, the vessel 500 may include a fluid filter 520, afluid pipe 521 and a fluid pump 522. In some embodiments, the dielectricfluid can be added to the vessel in a manner that causes liquiddielectric fluid to spill out of the rack 310 and into the sump area523. Fluid may then be filtered, using the fluid filter 520, and pumped,using the fluid pump 522 and fluid pipe 521, to the far side of thevessel. This system circulates fresh filtered dielectric fluid throughthe vessel, and thus, the dielectric fluid can be reused to cool thecomputing components.

FIG. 10D illustrates a section view of the vessel 500. In thisembodiment, a level of liquid dielectric fluid may be maintained at afluid level 524 above the height of the rack 310 and/or the computingcomponents therein. As a result, the rack 310 and/or the computingcomponents may be immersed in the dielectric fluid. Above the fluidlevel 524 may be saturated dielectric vapor, e.g., up to a halfway level525. In some embodiments, the saturated dielectric vapor is maintainedup to the halfway level 525, which may be at about half the height ofthe condenser coil 132. Above the saturated vapor is a headspace whichmay, in some embodiments, contain a less dense dielectric vapor.

In the example embodiment of FIG. 10D, the cooling coils 132 are locatedover a shelf area. As such, when the robot 526 places or removes thechassis 400, the cooling coils 132 are not in the way. This arrangementof the cooling coils 132 can streamline placement and removal of thechassis 400, thereby can offer significant benefits in autonomousoperation of the vessel.

The Communication System

Embodiments of the disclosed communication system are designed toprovide a standardized layer 1 through 3 connectivity and managementinterface for the equipment within or associated with the disclosedsuper structure 210, pressure controlled vessel 110, and/or computingsystem.

In some embodiments, a series of MTP interfaces provide the ability tobring multiple high density multimode fiber connections into thepressure controlled vessel 110. Once contained in the pressurecontrolled vessel 110, the fiber connections may be broken down toindividual switch level connections using a set of dedicated break outcables, break out interfaces, patch panels and/or distribution patchpanels to the racks 310.

Some embodiments of the disclosed system may include dedicated fiberpatch panel interface ports at each rack 310 to allow for connection tothe switches system installed therein via a short patch panel. In otherembodiments, there may be a dedicated patch panel, or set of patchpanels, running from each switch system to the MTP distributioninterface.

In some embodiments, the interface between the switch system and thechassis 400 may be via the backplane interface 330 and/or via some othermechanism which may or may not include the use of a backplane connector.In some embodiments, there may be no intermediary rack level switchsystem. Such embodiments may use a set of centralized switches withinthe pressure controlled vessel 110 to connect to various computingequipment located therein.

The typical interface between the switch system and the chassis 400 maybe accomplished using a patch panel attached to the rack 310 and wiredto the backplane system 330 with patch cables connecting the ports onthe patch panel with an appropriate port on the switch system.

In some embodiments, there will be a small (6U) rack rail areacontaining a patch panel interconnecting the communication systemcabinet with the MTP interfaces on each pressure controlled vessel 110,and centralized communication system distribution switches which serveto interconnect the switch system with each other and/or the outsideworld. In such embodiments, the end-user or customer may choose toeither install their own routing gear within this space and provideexternal connectivity thereto to serve as the connection between thedisclosed computing system and the outside world, or run fiberconnectivity between the pressure controlled vessel 110 or superstructure 210 and an existing network environment.

The access, communication, and/or networking components utilized withinan embodiment of the communication system environment may be standardequipment or may be user specified. The rack 310 and backplane interface330 systems may include the ability to replace the switch system locatedwithin each rack 310 by removing the existing switch, replacing it withany standard switch (such as a 1U switch), and re-wiring the desiredinterfaces to the backplane network interface panel.

In certain embodiments, products which are designed to interfacedirectly with the backplane system 330 may be utilized. Such productsmay utilize a chassis 400 patch panel system and/or a direct electricalinterface designed specifically to interconnect switch ports via aspecialized purpose built internetworking interface, via a commerciallyused protocol or via specification for the design of a network levelinterconnectivity interface.

In some embodiments, the connectivity between each blade or chassis andthe switch may contain multiple interfaces. One interface may be astandard switchport which may be the standard port which is available ona commercially available switch. A common interface can be 1GBASE-T or10GBASE-T which makes use of Cat6 or Cat7 twisted pair copperconnections between the switch and the host device. Another interfacemay be a switch-to-backplane intermediary device which may consist ofeither a patch panel with standard patch cables going from the standardswitchports to the front side of the patch panel and a set of hard wiredconnections from the back side of that patch panel to the signalinterfaces of the signal backplane. Alternatively, this could consist ofspecialized cables and/or standard RJ45 interfaces, which go from theswitchport to the board to establish connectivity between the standardswitchports and the backplane. Yet another interface may be an interfacesystem signal backplane which distributes the signal pathways from thestandard switchports along a printed circuit board (PCB). One or more ofthe signal pathways may be terminated at a connector on the PCB to whichthe signal backplane interface will connect. Yet another interface maybe a chassis signal backplane interface. This may be a connector locatedon the chassis itself which mates up to the connector on the interfacesystem signal backplane. It serves as the interface between theinterface system signal backplane with the chassis itself. Yet anotherinterface may be the chassis network interface. This may be a standardpatch interface allowing for the connection of a patch cable from thechassis network interface to the RJ45 interface on the server which isattached to the Chassis.

The Robotic Systems

In some embodiments of the disclosed system, a potential method ofaddressing the need for hot swapability of the components within thepressure controlled vessel 110. The need for the ability to remotelyremove and replace failed components 170 may be addressed throughrobotics.

A particular embodiment of the disclosed combination of systems mayinclude an internal robotic arm 230 and/or external robotic arm 240.Some embodiments, such as those for cryptocurrency applications and/orcertain high performance computing environments, may not require hotswapability of components. In other hyper scale GPU and CPUenvironments, this may be a fundamental requirement. Embodiments of thedisclosed robotic system allow for replacement of a chassis and/or othercomputer component without interruption of any other components. In someembodiments, failed cards and/or components may be automatically and/orprogrammatically replaced and/or stored. This allows for short andmid-term, fully remote and autonomous operation of embodiments of thedisclosed systems.

The internal robotic arm 230 mechanism is located within the pressurecontrolled vessel 110 environment. As shown in FIGS. 7A-D, in anexemplary embodiment, when a card or component is not operatingproperly, a removal sequence may be initiated. When a removal sequenceis initiated, the internal arm 230 will remove the appropriate computercomponent 170 and/or associated chassis 400 from the rack 310, move itto an airlock 220 located within the pressure controlled vessel 110, andsignal completion of the removal sequence. Once this sequence has beencompleted, the inner airlock door 222 will close, the airlock pressurewill be equalized with that of the outside atmosphere, and an exteriorairlock door 224 will open. Once the exterior door 224 has been opened,the external robotic arm 240 will remove the chassis 400 from theairlock 220 and place it into an empty storage slot.

In some embodiments, the airlock 220 will be purged with nitrogen and/oranother inert and/or non-condensable gas, before the airlock 220 isopened to the exterior environment. In such embodiments, this has theeffect of reducing or eliminating the loss of dielectric vapor when theairlock is opened and closed. In certain embodiments, the airlock willbe fit with one-way valves, on the interior portion, exterior portion,or both. In an embodiment with one-way valves on both the interior andexterior portion of the airlock, purging the airlock will prevent crosscontamination of the exterior environment into the interior atmosphereof the pressure controlled vessel 110 and also prevent loss ofdielectric vapor.

When a card or component replacement sequence is initiated, the externalrobot arm 240 will remove a replacement component and/or chassis 400from a storage slot and place the component into the airlock 220. Oncecompleted, the outer airlock door 224 will close, the airlock pressurewill be equalized with that of the inside of the pressure controlledvessel 110, and the inner door 222 will open. Once the inner door 222has been opened, the internal robotic arm 230 will remove the chassis400 from the airlock 220 and insert it into the appropriate rack 310.

When coupled with the remotely accessible management system, theinternal and external robotic arms 230 and 240 allow for the remoteoperation and management of a datacenter environment. This may reducethe need for human operators to remain available and reduce costs and/ordowntime. In some embodiment, the external robot arm 240 is mounted on amovable base, thereby allowing a single external robotic arm system toserve multiple embodiments of the disclosed computing system.

When integrated with custom developed workflow management systems andvirtualization technologies, the disclosed robotic systems allow for thedevelopment of completely autonomous, self-healing datacenter solutionswhich can provide maximum levels of system reliability.

In some embodiments, an asset tag with a unique human and/or machinereadable serial number and/or production batch code may be included oneach computer component, and/or chassis. In these embodiments, the assettag may be a unique serial number. The tag may contain a printed barcodeor QR code and allow for automatic part identification by embodiments ofthe disclosed robotic system. The tag code may also be used inconnection with a management software system to provide detailedcomponent information regarding inventory management and automationsystems. The tag and any associated adhesive or other components arepreferably made of a material that is compatible with the dielectricfluid. The tag is preferably located on the chassis in a spot that isreadable when the chassis is inserted into the rack. In someembodiments, secondary or additional tags may be located on other areaof the chassis to assist in identification of the components and/orinventory management.

Embodiments of the disclosed robotics system allow for any individualchassis to be temporarily removed and replaced, a process called“re-seating”. This is useful when during troubleshooting it isdetermined that a hard power cycle of the component is desired. A reseataccomplishes this by disconnecting all power, waiting a moment, andreconnecting it.

Some embodiments allow for an individual card and/or chassis to beremoved from the pressure controlled vessel through an airlock. In someembodiments, the robotics system will remove a chassis from its slot inthe rack, move it to an airlock, and signal the completion of this taskto allow for the airlock to be opened and the card and/or chassis to beremoved. Some embodiments allow for a replacement component and/orchassis to be placed into a particular rack slot through the sameairlock used for removal. In some embodiments, the robotic system willremove the chassis from the airlock, place it into the appropriate rackslot, and signal its completion of this task.

Robot on the Inside Systems

Embodiments of the disclosed system may contain a “robot inside”robotics system. In such embodiments, the pressure controlled vessel maybe expanded in order to accommodate a robotic arm operating within thevessel. The vessel may also be arranged to accommodate the movement ortransfer of computer components and/or chassis above the racks whichcontain operating computer components. It will be appreciated that apressure controlled vessel may also be referred to as the tank, pod,and/or vacuum chamber. Alternatively, it will be appreciated thatcertain components of the pressure controlled vessel may be referred toas the tank or pod.

FIG. 10E depicts an embodiment of the disclosed system with a gantryrobot 526 configured to remove, replace, and/or install computingcomponents, e.g., chassis 400 of the rack 310. In some embodiments, thegantry robot 526 may also be configured to remove, replace, and/orinstall DC rectifiers and/or other components of the power distributionsystem. It will be appreciated that some embodiments of the disclosedcomputer components and power distribution components may be designed tobe hot swappable and may include handles or other features whichfacilitate handling by the gantry robot 526. In some embodiments, thegantry robot 526 is arranged to travel in both x and y directions andmay drop down in the z direction in order to remove and/or installreplacement components. In some embodiments, the gantry robot 526comprises a gripping tool to grab the chassis 400 and/or power supplies,e.g., the gripping tool can grab the plate 426.

FIG. 10E illustrates a top section view of an exemplary embodiment ofthe disclosed tank. In some embodiments, an array of racks 310 may bepopulated with chassis 400 and/or computing boards. In some embodiments,each chassis 400 may utilize about 6 KW of power and each rack 310 maycontain 10 chassis. Accordingly, in embodiments which contain 10 of suchracks 310, the vessel may utilize about 600 KW of power for computingpurposes. In some embodiments, an additional rack 310 and/or magazine527 of chassis 400 and DC power rectifiers may be stored in the vessel500 to be used as replacement components and/or to provide a space tostore components that have been removed from the vessel 500.

Robot on the Outside Systems

FIGS. 12A-E show another embodiment of the vessel. In particular, FIG.12A depicts an embodiment of the vessel 700 in which the gantry robot526 is exterior of a tank 710 which houses the chassis 400 and/orcomputing components. In this embodiment, the tank 710 may be smallerbut will need to be opened more often for the external gantry robot 526to access the chassis 400 and/or power supplies inside the tank 710.Also, replacement equipment may be stored and/or housed within a modularenclosure such as the storage 716, which is outside of the tank 710. Insome embodiments, the tank 710 may have multiple doors 711, therebylimiting the exposure of the interior of the tank 710 when a single door711 is opened for the purpose of removing, installing and/or replacing acomponent or chassis 400. In such embodiments, replacement componentsmay be stored outside of the tank 710 in order to avoid opening the tankunnecessarily.

Additionally, the vessel 700 may include one or more transformers 712,electrical distribution panels 713, process water pipes 512 andelectrical chase 714. The vessel 700 may also include a programmablelogic controller (PLC) cabinet 715, which monitors and control thestatus of various equipment within the vessel 700. The transformers 712,electrical distribution panels 713, process water pipes 512, electricalchase 714 and PLC cabinet 715 may be located outside of the tank 710.

FIG. 12B illustrates a section view of the vessel 700 with the tank 710being accessible to the external gantry robot 526. In this exampleembodiment, the condenser coil 132, racks 310 and bellows 717 arelocated in the tank 710. FIG. 12C illustrates a side view of the vessel700 with the tank 710 with an external gantry robot and multiple doors711. In this example embodiment, the tank 710 includes a fluid pump forremoving fluid form a sump area and sending the fluid to a fluid filter520 through a fluid pipe 521. The vessel 700 also includes a magazine718 for storage of replacement equipment. In this example embodiment,the magazine 718 is located outside of the tank 710. In someembodiments, spacers and/or ballast blocks 160 may be used in order toreduce the total volume of liquid dielectric fluid in the tank 710.

FIG. 12D illustrates a rack 310 according to an example embodiment. Insome embodiments, a redundant power supply 314 may be positioned on theopposite side of the rack 310 as opposed to adjacent to the primarypower supply 313. Additionally, power and/or data cables 318 and 319 maybe routed in alternative configurations in order to accommodate thespecific requirements of a particular deployment. In this exampleembodiment, a backplane receiver 331 is located under the rack 310.

FIG. 12E shows an example hinging door 711 that may be used in somealternative embodiments of the disclosed tank 710. In some embodiments,sliding doors, as opposed to hinged doors may be utilized in order toreduce or avoid inducing currents in the dielectric vapor. Slowlysliding a door open will likely disturb the dielectric vapor less thanswinging open a hinged door and causing mixing currents.

The Management System

The management system is a web based interface between the user of thedisclosed computing system and the computing system itself. Embodimentsof the management system provide an operational view of the computingsystem and allow for the monitoring and management of the variouscomponents, including monitoring and managing the pressure controlledvessels 110, the robotic systems, the communication system, the powersystem, and/or all other systems and components. In one exampleembodiment, the management system may be implemented in the PLC cabinet715 of FIG. 12A. In another example embodiment, the management systemmay be implemented in the power and communication box 513 of FIG. 10A.in each embodiment, the power management system can be implemented as asoftware program on a control device or other suitable device, e.g., acomputer.

In certain embodiments, a set of simple network management protocolaccessible data points may be made available to users of the managementsystem to allow for monitoring of key operational parameters via thirdparty monitoring systems. Full operational logs may be maintained andcharts may be provided for user review of operational condition data.

Regular maintenance of system components may be scheduled and maintainedvia the management system. The user may be provided with regularreminders of routine maintenance and be able to acknowledge those asbeing performed within the interface. This data may all be maintained aspart of the operational log information for historical operationalreview.

In some embodiments, operational functionality may also be exposed viaan API interface to allow for the remote programmatic monitoring andmanagement of the computing system and associated components. A full setof operational monitoring and alerting functionality may be included toallow for the notification of operators in the event of any issues.

A centralized server version or a hosted cloud based management versionof the management system may be utilized by customers with multiplepressure controlled vessel computing systems. This provides the operatorwith a single programmatically and user accessible interface for themanagement of a fleet of pressure controlled vessel computing systems.

In some embodiments, software based interface modules allow forinteroperation with the computing platform and third party managementutilities, such as Microsoft System Center and VMWare VCenter. The userand API interfaces provided by the management system may allow completeinteroperation with the disclosed robotic systems, allowing for completeremote and programmatic autonomous operation and administration of thedisclosed computing platform.

In some embodiments, control systems allow for adjustment and control ofoperations including temperature, pressure, flow rate, and/or powermanagement. In some embodiments, a user authentication system allowsmultiple unique users to authenticate to the system. Some embodimentsinclude role based and/or element based permission systems. In suchembodiments, an administrator will be able to configure multiple roleswith which users may be associated and/or apply specific permissions toindividual users outside of their role allocations.

Some embodiments incorporate video management in order to provide userswith the ability to record and retrieve video feeds from cameras whichmay be located within a vessel and/or super structure. In someembodiments, the cameras may acquire visual data which may be analyzedby a processor. In such embodiments, the processor may utilize computervision techniques in order to control operations of the vessel,robotics, and/or super structure systems in response to the acquiredvisual data.

In some embodiments, the control system and software may be configuredto generate reports regarding the operations and status of the overallsystem and/or the individual subsystems and/or components of thedisclosed computing platform.

Exemplary Combined System Embodiments

It will be understood that the disclosed systems may be utilizedindividually or in combination. There are multiple embodiments of thecombined computing system which may be tailored to various use cases.

One exemplary embodiment is the Crypto Series. This is an ultra-highdensity implementation of the disclosed technology utilizing purposebuilt computing hardware, racks 310 with guide plates and wiringharnesses designed for that hardware, a modified implementation of thecommunication system 360 architecture, and a 1 MW pressure controlledvessel 110 and power distribution system. The typical user of thisembodiment are those who wish to perform cryptocurrency mining or otherultra-high power density processes using customized computingcomponents, or manufacturers of computing components who wish to developa full scope two-phase liquid immersion cooling system into which theywill incorporate their own hardware.

Another exemplary embodiment is the GPU Series. This is a high densityGPU supercomputing implementation of the disclosed technology. Thisimplementation will make use of custom made chassis 400, rack 310 andbackplane interface 330 technology designed to incorporate motherboardsfrom Gigabyte and GPUs from NVidia using the NVidia NVLink technology tofacilitate ultra-high speed GPU to GPU communications. The typical usersof this technology include general purpose parallel processingapplications which can make use of the GPU based computing and memorycapabilities, including graphics rendering, particle simulation andgeneral research activities.

Yet another exemplary embodiment is the CPU Series. This is a highdensity CPU computing implementation of the disclosed technology. Thisimplementation will make use of high end Supermicro based motherboards,Intel Xeon CPUs, high speed network interfaces, high speed memory, andsolid state storage devices for local storage. The typical user of thistechnology includes datacenter, enterprise, and cloud/VPS hostingproviders and service providers who utilizing high performance computingfor either their own internal applications or for those which theyprovide to third party customers and other organizations.

Still another exemplary embodiment includes the Edge Series. This is ascaled down implementation of the disclosed computing system which isdesigned specifically for remote/field deployments or within oralongside traditional business and data center environments. Theembodiment is focused on a secure, weatherized environment with fullremote monitoring and management capabilities. The target users of thistechnology would be operators of field deployed and distributedtechnologies, such as network operators and other organizations withdistributed field infrastructure, and operators of existing facilitieswishing to augment their computing capabilities with minimalmodification to existing facilities or structures. This system mayincorporate various enhancements to the external structure to simplifythe connection of utility service, including electrical, water andnetwork connectivity to the system.

Self-Contained Embodiments

Some disclosed embodiments do not rely on an external source of water.Such embodiments may comprise a closed-loop chiller for cooling water oranother fluid which may be circulated through a condenser as describedabove. The use of a closed-loop chiller, rather than an external sourceof cooling water allows for embodiments which are substantiallyself-contained.

FIG. 13 shows an example self-contained vessel 750. The exemplaryembodiment of FIG. 13 utilizes a skid-mounted closed-loop chiller 719for cooling water or another liquid to be used in a condenser within thepod or immersion tank 710. By utilizing a closed-loop chiller, the needfor an external source of cooling water may be eliminated resulting in aself-contained data center solution which may require only an externalsource of power and a network connection in order to be fullyoperational. The vessel 750 may also include bellows 717, door 711,gantry robot 526, electrical distribution panel 713, PLC cabinet 715 andmagazine 718.

In some embodiments, the closed-loop chiller 719 may be a skid-mountedclosed loop chiller which is enclosed within an outer housing of amodular pressure controlled vessel. In such embodiments, heat istransferred from computer components to a dielectric liquid within thetank 710. This process converts the dielectric liquid to a dielectricvapor as discussed herein. The dielectric vapor rises within the tank710 and is cooled by a condenser, thereby converting the dielectricvapor back into a dielectric liquid. The heat transferred from thedielectric vapor to the condenser is then transferred from the condenserto a refrigerant or condensation fluid within the condenser and then toa closed-loop chiller 719. In some embodiments, the chiller 719 removesheat from the refrigerant or condensation fluid using vapor-compression,a compressor, an evaporator, a heat-exchanger or other closed-loopmethod of cooling the refrigerant or condensation fluid. The heat fromthe refrigerant or condensation fluid is ultimately dispersed via aircooling. In some embodiments, this results in a self-contained, modular,air-cooled, two-phase liquid immersion computing system. The air coolingof some self-contained embodiments is surprising as the field ofimmersion cooling has commonly taught against air cooling, especiallyair cooling of a self-contained device.

Some disclosed embodiments may be provided in a form factor with aspace-saving footprint. An exemplary embodiment comprises a single rackcontaining ten blades or servers immersed in a dielectric liquid asdescribed above. In some embodiments, each server may draw about 6 kW ofpower. Accordingly, some exemplary embodiments provide about 60 kW ofcomputer power in a small footprint.

The exemplary embodiment illustrated in FIG. 13 is contained within afoot print that is about 4 feet 2 inches deep, about 8 feet 8.5 incheswide, and about 8 feet 8 inches tall. This exemplary embodimentcomprises about 60 kW of computer power as well as the other operationalcomponents and systems and is contained in an area of about 36.3 squarefeet. It will be appreciated that operational components of a vessel mayinclude, but are not limited to, a tank or pod containing a dielectricfluid, condenser, power supply, and data connection for the computercomponents. The vessel may also comprise sensors, control equipment, apower cabinet, a bellows 717, a vacuum system, fluid filter, purgesystem, and/or other components. Some self-contained embodiments maycomprise an outer housing. In some embodiments, the outer housing mayenclose the vessel, provide structural support, be skid mountable, beventilated, be weather and/or water resistant, and/or be decorative. Insome embodiments, the outer housing of a self-contained vessel maycomprise a radiator coil, fan grate, thermal transfer components, and/orair-cooled components to facilitate the use of a closed-loop chiller.

In some embodiments, a self-contained computing system provides at leastabout 1.5 kW of computing power per square foot of area, at least about1.6 kW per square foot, at least about 1.65 kW per square foot, at leastabout 1.8 kW per square foot, at least about 2.0 kW per square foot, orat least about 3.0 kW per square foot. In some embodiments, aself-contained computing system provides at most about 1.5 kW ofcomputing power per square foot of area, at most about 1.6 kW per squarefoot, at most about 1.65 kW per square foot, at most about 1.8 kW persquare foot, at most about 2.0 kW per square foot, or at most about 3.0kW per square foot. It will be appreciated that the height of aself-contained system may be adjusted, thereby allowing for more or lesscomputing power to be provided within a given footprint.

It will be appreciated that the dimensions, components, arrangement, andconfiguration of the disclosed exemplary embodiments may be modified,added to and/or subtracted from to produce a variety of potentialembodiments in a variety of form factors.

In some embodiments, a self-contained computing system may comprise arobotic system, such as, for example, a gantry robot 526, configured toremove, replace, and/or install blade servers, power supplies, or othercomponents, e.g., chassis 400. A self-contained system may compriseeither a “robot on the inside” or a “robot on the outside” system. Inembodiments with smaller footprints, a smaller magazine 718 ofreplacement components may be used. In some embodiments, the magazine718 of replacement components may be attached to the exterior of thetank 710 as shown in FIG. 13. In some embodiments, tank 710, rack,computer components, power supplies, replacement magazine 718, andgantry robot 526 may be arranged such that the gantry robot 526 mayremove, replace, and/or install components while traveling insubstantially only one direction. If the various components are arrangedsubstantially linearly, the gantry robot 526 may be able to travel alonga single axis in order to remove, replace, and/or install the desiredcomponents without traveling in a second direction. It will beappreciated, that the gantry robot 526 may be able to lift and lowercomponents in addition to traveling in a single linear direction.

Utilizing a compact form factor, such as the embodiment illustrated inFIG. 13 allows the self-contained 2PLIC system to be easily transported.The incorporation of a closed-loop chiller 719 allows for two-phaseliquid immersion cooling system to be utilized in remote conditionswhich may not have access to a practical source of chilled water.Additionally, the elimination of a need for external cooling watercreates a self-contained computing system which, in some embodiments,requires only two external connections, a power supply and a dataconnection.

In some embodiments, the computing system may be contained within anouter housing as shown in FIG. 14. In some embodiments, the componentsschematically identified in FIG. 13 and/or disclosed herein may becontained within the outer housing. In some embodiments, the volume ofthe outer housing may be adapted based on the anticipated coolingrequirements, configuration of a closed-loop chiller and/or theenvironment in which the self-contained computing system is expected tobe deployed.

The disclosed self-contained, self-healing, compact form factorembodiments may be used as a stand-alone solution to provide significantcomputing capabilities to almost any location or environment. In someapplications, multiple compact computing systems may be positioned neareach other and/or linked together to create a cluster. In someembodiments, the outer housing is arranged to allow maintenance and/orservice operations to be performed while accessing only one or two sidesof the outer housing. This arrangement allows the individualself-contained computing systems to be positioned with reduced orminimal distance between each self-contained system.

In one example embodiment, clusters of four exemplary self-containedcomputing systems may be positioned strategically to allow for about 240kW of self-contained computer power in about a 140 square footfootprint. In some embodiments, these units may be in power and/or datacommunication with each other, thereby allowing for the operation of amulti-unit cluster with only a single external power connection and asingle data connection. In some embodiments, a data center may beestablished using multiple compact computing systems or multipleclusters of such computing systems.

Some disclosed embodiments and/or computing systems disclosed herein maybe utilized in modern data centers and/or climate controlledenvironments, however some embodiments of the disclosed self-containedcomputing system may be deployed in remote locations and/or harshenvironmental conditions. In some embodiments, the outer housing may beweatherized, waterproof, and/or otherwise arranged to tolerate exposureto harsh environments for an extended period of time. Some disclosedembodiments allow for the rapid deployment of significant computingresources to remote or challenging locations. Some self-containedembodiments may be arranged to be operational in substantially anylocation with access to a power supply and data connection. In someembodiments, an uninterruptable power supply and/or generator may beoperably connected to the computing system to provide more reliable orconsistent access to electrical power.

Some disclosed self-contained embodiments are designed to be stackable.Some stackable embodiments may be designed with a reduced height.Certain embodiments may be about 5′5″ tall, 5′6″ deep and 9′ wide. Themay result in about 60 kW of computer power in a 42 square footfoot-print. Such units may be vertically stacked to provide 120 kW ofcomputer power in the same 42 square foot foot-print.

Embodiments of the disclosed computing system may be stacked andmultiple stacks may be positioned adjacent to each other. This reducesthe need for isle space between individual computing systems, therebyallowing for an overall higher power density within a data center.

Some embodiments may be designed to be fully operable and maintainablewith access to only one side of self-contained computing system. Suchembodiments may be advantageous as they facilitate positioning theself-contained systems in very close proximity to each other.Additionally, in some self-contained embodiments, the entire immersiontank may be removed and/or replaced while accessing only one side of thedevice. In certain embodiments, the tank may be individually modularand/or skid mounted.

In some embodiments, a self-contained computing system may be arrangedvertically to utilize an even smaller footprint. A vertically designedembodiment of the disclosed system may provide about 60 kW of computingpower in about a 22.9 square foot foot-print. As with some otherdisclosed embodiments, some vertically oriented self-contained computingsystems may be positioned in close proximity to one another. Also asdiscussed with some other embodiments, some vertically orientedself-contained computing systems may be operated and maintained withaccess only to one side of the device. In some embodiments, the entiretank may be removed from the outer housing and replaced. Thisarrangement allows for the rapid replacement of multiple blade serversand/or other computing components.

Mobile Embodiments

Self-contained computing systems which do not require an external sourceof cooling water allow for novel computing applications. In someembodiments, power may be provided to the system using a generatorthereby removing the need to connect the system to a source of externaland/or stationary power. In some embodiments, the system may rely onwireless data communication.

In particular self-contained embodiments which do not rely on astationary power source or wired data communication, a fully mobilecomputing system may be implemented. Disclosed embodiments includevehicle mounted, self-contained computing systems which may be used toprovide significant computing power in nearly any environment. In someembodiments, a truck-mounted, wireless computing system may be drivenwithin wireless communication range of an existing or temporary networkand provide a significant amount of computing power with substantiallyno setup or installation time.

Natural Water Embodiments

In some embodiments, a computing system may be arranged to be utilizedon a boat, ship, oil-rig, floating platform, or other vessel orstructure which is located in close proximity to a body of water. Insuch embodiments, the condenser, used to convert dielectric vapor backto a dielectric liquid as discussed herein, may be cooled using waterfrom the body of water. In one exemplary embodiment, a modular computingsystem may comprise a water intake, water output, and pump or impeller.The pump and/or impeller may cause water to flow from the body of water,through the condenser, and then back into the body of water. Someembodiments may comprise filters and/or process components designed toprotect condenser, piping, and other computing system components frompotential sources of contamination in the body of water. In someembodiments, the condenser and other components are arranged to endureextended contact with brackish or salty water such as, for example,ocean water.

Horizontal Magazine Swap

In some embodiments, a magazine of replacement components may be storedon the outside of a tank and within an outer housing of a computingsystem. Replacement components, such as, for example, chassis, servers,blades, and/or power components, may be removed from the magazine andused to replace components within the tank. The magazine may be on aplatform which is configured to extend out of the outer housing of thecomputing system in order to allow components from the magazine to bereplaced.

In one non-limiting example, when a blade server within the tankmalfunctions, a robotic arm may be used to extract the non-workingcomponent from the tank and move the non-working component to a storageslot of a magazine. The robotic arm may then remove a functioning bladeserver from the magazine and install it where the non-working server waspreviously installed, thereby replacing the non-working server with anew working server.

Over time, the magazine will accumulate non-working components which maybe replaced with new working components in order for the robotic systemto continue long term operations. In some embodiments, the magazine ison a platform which may extend to the outside of the outer housing,thereby allowing an operator to access the magazine. In someembodiments, the platform is configured to rotate the magazine from asubstantially vertical position to a substantially horizontal positionin order to allow components to slide in or out of the magazine.

In some embodiments, an adjustable height cart may be used to move,load, and/or receive components so that a human operator is not requiredto lift or support the weight of the components while removing orreplacing components from the magazine. It will be appreciated that amagazine that is configured to rotate to a substantially horizontalposition may also facilitate loading of functioning components into themagazine as well as removing non-functioning components.

FIGS. 15A-D show an example magazine 810 located on a platform 820capable of extending out of the vessel. In FIG. 15A, the magazine 810may be connected to a platform including a rotating member 821, asupporting member 822 and a rail 823. In some embodiments, thesupporting member 822 may be connected to the rail 823 that allows thesupporting member 822 to move while supporting the weight of themagazine 810 and any servers or other components stored within themagazine. In the example embodiment of the FIG. 15A, the platform 820 isin an extended position.

As shown in FIG. 15B, during normal operations, the supporting member822 may be retracted with respect to the outer housing of the computingsystem. The magazine 810 may be stored above the rail 823 during normaloperations. In some embodiments, the weight of the magazine 823 issupported by the supporting member 822 and rail 823 regardless of theposition of the supporting member 822 on the rail 823.

In some embodiments, computer components, such as, for example, servers,utilized with disclosed embodiments may be denser and/or heavier thantraditional computer components. In some embodiments, due to theincreased cooling capabilities of disclosed embodiments, a blade servermay weigh at least as much as about 50 lbs, or at least as much as about60 lbs, or at least as much as about 70 lbs, or at least as much asabout 80 lbs, or at least as much as about 90 lbs, or at least as muchas about 100 lbs. In some embodiments, a blade server may weigh at mostas much as about 50 lbs, or at most as much as about 60 lbs, or at mostas much as about 70 lbs, or at most as much as about 80 lbs, or at mostas much as about 90 lbs, or at most as much as about 100 lbs. As shownin FIG. 15B, a magazine 810 can hold a plurality of chassis 400 or bladeservers where an individual blade server may weigh about 73 lbs. Whenthe magazine is loaded with three such servers, the combined weight ofthe magazine 810 and servers may be about 395 lbs.

In some embodiments, the servers used are blade servers mounted on achassis. The server and/or chassis may contain a backplane system tofacilitate the installation and removal of the servers in the computingsystem. In some embodiments, the servers may be immersion servers whichdo not include a fan or other air cooling devices. In some embodiments,an individual server board may comprise 16 GPU's and be configured todraw about 6 KW of power. In some embodiments, the servers are 1.5Uservers. Some disclosed servers may be 1 Otto Immersion Unit (OIU)servers. Such servers are 1.5U tall and configured for liquid immersioncooling. In some embodiments, a single tank within the computing systemmay be configured to operate ten 1OIU servers and about 60 KW of powerwhen all ten severs are operating at substantially full power. In someembodiments, the computing system may comprise one or two of such tanks.In some embodiments, the computing system may comprise multiple tankssuch as, for example, ten such tanks.

In some embodiments, when a magazine is extracted from the computingsystem, as shown in FIG. 15A, the supporting member moves along the railfrom a storage position and is cantilevered outside of the outer housingof the computing system.

As also shown in FIGS. 15C-D, the magazine may be pulled out orotherwise slide along the rails and be cantilevered outside of thecomputing system. In some embodiments, as shown in FIGS. 15C-D, amagazine removal tool may be used to remove the entire magazine and thecomponents contained within the magazine. In such embodiments, themagazine removal tool may be used to lift the magazine off of thesupporting member and slide rails in order to transport the magazine.

In some embodiments, once the magazine has been moved to the exterior ofthe computing system, the platform may rotate the magazine to asubstantially horizontal position. The servers contained within themagazine may then slide out of the magazine.

FIGS. 15A-D illustrate an exemplary series of steps for removing aserver from a magazine according to an exemplary embodiment. In theexemplary embodiment, the magazine may be attached to a linear guiderail system behind an access door. As shown in FIGS. 15C-D, the magazinemay be pulled out and cantilevered outside the computing system. Themagazine may be pulled out manually or be moved out of the computingsystem using a motorized or automated system. As shown in FIG. 15D, themagazine may be rotated about 90 degrees to orient the servers and/orother components contained in the magazine in a substantially horizontalposition. Once in a substantially horizontal position, the serversand/or other components can slide out of the magazine and onto a cart orother tool configured to receive the servers and/or other components. Asshown in FIG. 15C, a scissor-lift cart may be adjusted to a convenientheight to receive the servers or other components. An adjustable heightcart with a rolling surface may be used to allow the servers to betransferred from the magazine onto the cart without requiring a humanoperator to support the weight of the servers. As shown in FIG. 15D,once a server is slid onto a cart with a gliding or rolling surface, theserver or other component may be transported elsewhere to be replaced orserviced. It will be appreciated that new components may be loaded intothe magazine using substantially the same steps in the reverse order.

In some alternative embodiments, the magazine may be supported on arotating and extendable arm without a rail. In such embodiments, themagazine may be stored in a substantially vertical position within theouter housing of the computing system during normal operations. Once itis determined that the components within the magazine should bereplaced, the magazine may be extended outside of the outer housingusing the extendable arm. Once the magazine is extended beyond the outerhousing, the magazine may be rotated from a substantially verticalposition to a substantially horizontal position to allow the componentsstored within the magazine to be horizontally removed from the magazine.

Bellows

In some embodiments, a bellows and/or vapor collection system may beutilized. Before some disclosed embodiments are initially activated thedielectric fluid, computer components such as servers, and other systemcomponents may be at thermal equilibrium. Once a computing system isactivated, the computer components, such as servers may begin togenerate heat that maybe dissipated into the dielectric fluid. Thisprocess causes some of the dielectric fluid to shift from a liquid stateto a vapor state. As the temperature of the fluid increases, a greaterproportion of the dielectric fluid may shift to a vapor state. In aclosed system, the increasing volume of dielectric vapor may result inincreased pressure within the system. In some embodiments, the tankcontaining the dielectric fluid may be in fluid and/or vaporcommunication with a recovery system.

FIG. 16 shows a vapor recovery system 900 according to an exemplaryembodiment. The recovery system 900 is connected to the tank 710containing dielectric vapor. Dielectric vapor may flow from the tank 710through piping to one or more bellows 905. In some embodiments, thevapor recovery system 900 comprises an expanding and collapsing bellows905 configured to receive the dielectric vapor, thereby reducing oreliminating any pressure build-up in the tank 710. When the system coolsor a portion of the dielectric vapor is condensed to dielectric liquid,the bellows may collapse or contract to substantially maintain apressure equilibrium within the tank 710.

In some embodiments, the vapor recovery system 900 comprises a valve 912configured to allow ambient air into the vapor recovery system. In suchembodiments, the dielectric vapor may be mixed with ambient air. Mixingdielectric vapor with ambient air may reduce the temperature of thedielectric vapor. In some embodiments, the mixed air/vapor may bedirected through a carbon bed 911. The carbon media within the carbonbed 911 may be configured to attract the dielectric vapor while allowingthe ambient air to pass through the carbon media and be vented from thesystem 900, e.g., through an outlet valve 913. In this manner, theheated dielectric vapor may be cooled and captured by the carbon media.

Upon operating for a sufficient period of time, embodiments of thecomputing system will reach a steady thermal state based on the powercapacity of the computing components being utilized. If more or lesscomputing power is utilized, more or less dielectric fluid may beshifted to dielectric vapor. This may cause the bellows 905 to inflateand/or deflate in response to the heat dissipated into the dielectricfluid.

In some embodiments, the bellows 905 may comprise one or more pouches.Each pouch may comprise a metal foil and polymer laminated construction.The bellows pouches may be connected to the vapor recovery system pipingand to each other in series or in parallel. In some embodiments, thetotal volume of the expanded bellows pouches may be at least about 15%of the liquid fluid volume of the tank. In some embodiments, the totalvolume of the expanded bellows pouches may be at least about 20%, or atleast about 23%, or at least about 25%, of the liquid fluid volume ofthe tank or greater. In some embodiments, the total volume of theexpanded bellows pouches may be at most about 40%, or at most about 30%,or at least about 25%, of the liquid fluid volume of the tank or less.

In some embodiments, when the computing system has substantially reachedthermal stabilization, the vapor recovery system 900 may be closed tocooling ambient air and a valve allowing the air to be exhausted out ofthe system may be closed. In some embodiments, the carbon bed may beconfigured to be opened to only the tank and bellows using valves. Insome embodiments, a desorption heater configured to communicate heat tothe carbon media may be activated to raise the temperature of the carbonmedia. As the carbon media temperature increases, any dielectric fluidpreviously captured by the carbon media may be driven off of the carbonand back into the tank where it can be condensed back to dielectricfluid as previously described.

In some embodiments, when a computing system is powered at less than aprevious steady state, the portion of dielectric fluid in the vaporstate may be reduced and, in some embodiments, the bellows may contractto accommodate the reduction in dielectric vapor. In some embodiments, avalve that allows ambient air into the bellows may be opened in order toallow air into the bellows and further reduce any pressure differential.In some embodiments, nitrogen, rather than ambient air, may be used toreduce a pressure differential and also avoid introducing any potentialcontamination from ambient air.

In some embodiments, the bellows and/or vapor recovery system can becompletely, or substantially, passive. In some embodiments, the bellowsand/or vapor recovery system can be powered and/or automated based onsensor data from temperature, pressure, and/or power sensors positionedthroughout the computing system.

In some embodiments, the computing system with vapor recovery system isemission free even if the system is not a closed system. In someembodiments, ambient air or nitrogen may be introduced into the systemand exhausted out of the system without releasing any or substantiallyany dielectric fluid into the surrounding atmosphere.

EXEMPLARY EMBODIMENTS

Disclosed embodiments allow for increased density of computer componentsand/or computing power. In some embodiments comprising two-phase liquidimmersion cooled computer components 170 within a pressure controlledvessel 110, components may be separated from each other by less thanabout 1″ or less than about 0.7 inches, or less than about 0.5 inches.In some embodiments, individual components may be separated by more thanabout 0.3 inches, or more than about 0.5 inches, or more than about 0.7inches, or more than about 1 inch, or more than about 1.5 inches.

Some disclosed embodiments allow improved power utilizationeffectiveness (PUE) as compared to a traditional data center. Usingdisclosed embodiments allows for reduced energy usage for coolingcomputer components 170, thereby reducing the total energy usage of adata center and bringing the PUE closer to 1.0. Some embodiments relateto data centers comprising two-phase liquid immersion cooled computercomponents within a pressure controlled vessel 110 wherein the datacenter has an PUE of less than about 1.15, or less than about 1.10, orless than about 1.08, or less than about 1.05. Some embodiments relateto data centers comprising two-phase liquid immersion cooled computercomponents within a pressure controlled vessel 110 wherein the datacenter has an PUE of more than about 1.05, or more than about 1.06, ormore than about 1.08, or more than about 1.10.

In some embodiments a thermally conductive condensable dielectric fluidis provided to be used in a two-phase liquid immersion cooling system.Computer components are operated under less than ambient atmosphericpressure which reduces the temperature at which the dielectric fluidvaporizes, thereby maintaining the liquid phase of the dielectric fluidat a lower temperature as compared to standard atmospheric pressure. Thecomputer components generate heat as they operate. The generated heat istransferred to the dielectric liquid in contact with the computercomponents, causing the dielectric liquid to vaporize into a gas. Thegaseous dielectric fluid may be condensed using a condenser. Ambienttemperature, or chilled process water is passed through the condenser.When the gaseous dielectric fluid is cooled by the condenser, itcondenses back to a liquid phase and falls back into a bath of liquiddielectric fluid.

Some disclosed embodiments relate to high density data centers.Traditional data centers include about 1 megawatt (MW) of computingpower dispersed over about 10,000 square feet. High end data centers mayinclude about 1 MW of computing power dispersed over about 6,000 squarefeet. Disclosed embodiments relate to data centers comprising two-phaseliquid immersion cooled computer components 170 within a pressurecontrolled vessel 110 wherein the data center utilizes about 1 MW ofcomputing power dispersed over about 3,000 square feet, or about 1,500square feet, or about 1,000 square feet, or about 800 square feet, orabout 600 square feet. In some embodiments, multiple pressure controlledvessels containing the disclosed computing system may be arranged inrows and powered by a central power supply. In some embodiments,multiple embodiments of the disclosed computing system may be connectedto each other in series.

Disclosed embodiments comprise liquid immersion cooled computercomponents 170 within a pressure controlled vessel 110, accordingly, thecomponents are insulated from atmospheric contamination by the pressurecontrolled vessel and by being submerged in dielectric liquid 140. Somedisclosed embodiments relate to data centers which operate with minimalair filtration and/or cleaning requirements. In some embodiments, thedata center operates in the absence of HEPA filters or the equivalent,or in the absence of MERV 11 filters or the equivalent, or in theabsence of MERV 8 filters or the equivalent.

Disclosed embodiments comprise liquid immersion cooled computercomponents 170 within a pressure controlled vessel 110, accordingly, thecomponents are not cooled by air of gases. Disclosed embodiments includea data center which operates in the absence of cooling fans and/or anyother similar device for circulating air.

Disclosed embodiments relate to environmentally friendly data centers.In some embodiments, a data center comprise liquid immersion cooledcomputer components 170 within a pressure controlled vessel 110 andconsume little to no water for cooling processes. Some embodimentsutilize a closed circuit dry cooling tower to reduce the temperature ofwater which is circulated through the disclosed condensing structures130 in order to cool the condensing structures 130 and condensedielectric fluid vapor into dielectric fluid liquid. Such embodimentsoperate without significant input or output of water as the closed loop,dry cooling tower does not rely on evaporative cooling or a stream ofwater for cooling operations. Some data center embodiments utilizeand/or discharge less than about 10,000 gallons of water per day, orless than about 1,000 gallons of water per day, or less than about 100gallons of water per day, or less than about 10 gallons of water perday, or 0 gallons of water per day. Some data center embodiments utilizeand/or discharge more than about 100 gallons of water per day, or morethan about 1,000 gallons of water per day, or more than about 10,000gallons of water per day.

Disclosed embodiments relate to a computing system comprising a pressurecontrolled vessel operably connected to a pressure controller and/orsource of vacuum, wherein the pressure controlled vessel has an interiorand an exterior and is configured to contain an atmosphere within theinterior; a volume of thermally conductive, condensable dielectricfluid; a rack for mounting computer components, wherein the rack isarranged such that the computer components are at least partiallysubmerged within the volume of thermally conductive dielectric fluidwhen mounted on the rack; and a condensing structure, wherein the volumeof thermally conductive dielectric fluid, rack, computer components, andcondensing structure are contained within the pressure controlledvessel. Some embodiments relate to a cooling system comprising apressure controlled vessel comprising an interior wherein said vessel isconfigured to be operably connected to a pressure controller to reducethe interior pressure below atmospheric, wherein the pressure controlledvessel is configured to comprise a volume of thermally conductive,condensable dielectric fluid in liquid and gas phase; one or morecomputer components arranged such that the one or more computercomponents may be at least partially submerged within the liquid phaseof a volume of thermally conductive, condensable dielectric fluid; and acondenser for condensing gas phase dielectric fluid to liquid phasedielectric fluid.

In some embodiments, the pressure controlled vessel is mounted within asuper structure, the blade server is configured to be swappable withoutdisruption of the computing system, the pressure controlled vessel isoperably connected to a power supply, source of water, and networkingconnections, the pressure controlled vessel comprises an opening on thetop and a lid configured to sealably close the opening, the lid isconfigured to direct rising vapors from the middle of the pressurecontrolled vessel to the sides of the pressure controlled vessel, thepressure controlled vessel has an interior volume of between about 100cubic feet and about 300 cubic feet, and/or wherein the pressurecontrolled vessel contains a ratio of liquid dielectric fluid to gaseousdielectric fluid of between about 1:3 and about 1:8. Some embodimentsfurther comprise a ballast block, a blade server and a blade serverchassis, a robotic arm and an airlock, wherein the airlock is configuredto allow access to the interior of the pressure controlled vesselwithout significantly disrupting the atmosphere within the pressurecontrolled vessel, and/or a purge system, wherein the purge system isconfigured to remove contaminants from the volume of thermallyconductive dielectric fluid. In some embodiments, the purge system isconfigured to remove a portion of the atmosphere from the pressurecontrolled vessel, condense any dielectric fluid from the atmosphere,and discard any remaining vapors. In some embodiments, the purge systemis configured to condense at least a portion of gaseous dielectricfluid, and discard gaseous contaminants.

Some embodiments relate to a method for cooling computer components, themethod comprising: providing a housing, wherein the housing contains athermally conductive, condensable dielectric fluid and heat-generatingcomputer components, and wherein the housing is configured to withstandat least a slight vacuum; operating computer components, whereinoperating the computer components generates heat and wherein thecomputer components are in contact with the dielectric fluid; andcreating a vacuum within the housing, wherein the pressure within thehousing is at least below about 1 atmosphere. Some embodiments furthercomprise maintaining the vacuum within the housing, wherein the pressurewithin the housing is below about 1 atmosphere while the computercomponents are operating, vaporizing the dielectric fluid from a liquidstate to a gaseous state using the heat generated by the computercomponents and condensing the dielectric fluid from a gaseous state to aliquid state using a condenser, removing fluids which are not readilycondensable from the dielectric fluid. In some embodiments, and/orreplacing a portion of the computer components while the system isoperating. In certain embodiments removing non-condensable fluidscomprises isolating a portion of the gaseous atmosphere from within thehousing, condensing any dielectric fluid from the gaseous atmosphere;returning the condensed dielectric fluid to the housing, and discardingany remaining portion of the gaseous atmosphere and/or the housing isconfigured to generate a convection current.

Some embodiments relate to a method of cooling computer componentscomprising operating computer components at less than ambient pressure,wherein the computer components are in contact with a thermallyconductive, dielectric fluid. Some embodiments further comprisevaporizing the dielectric fluid and condensing the dielectric fluid atless than ambient pressure.

Some embodiments relate to a method for cooling computer components, themethod comprising: providing a thermally conductive, condensabledielectric fluid in a liquid and gas phase; and operating computercomponents at a pressure below ambient atmospheric pressure in thepresence of the thermally conductive, condensable dielectric fluid,wherein the computer components are at least partially in contact withthe thermally conductive, condensable dielectric fluid in liquid phase.Some embodiments further comprise vaporizing the dielectric fluid from aliquid phase to a gas phase using at least a portion of any heatgenerated by the operating computer components; condensing at least aportion of the dielectric fluid from a gas phase to a liquid phase;removing at least a portion of non-readily condensable fluids from thedielectric fluid; and/or replacing at least one or more computercomponents while said computer components are operating.

Some embodiments relate to a method of cooling computer components, themethod comprising: operating computer components at least at 1 psi lessthan ambient pressure, wherein the computer components at leastpartially in contact with a thermally conductive, dielectric fluid, andwherein the boiling point of the dielectric fluid is below about 80° C.Some embodiments further comprise condensing the dielectric fluid atconditions such that the computer components do not exceed about 80° C.

It will be understood that the various disclosed embodiments mayincorporate some or all of the components otherwise described herein.The particular components and the properties thereof may be adjustedbased on the properties of each particular embodiment. Modifications mayinclude the use of higher or lower density power, cooling, and networkconnectivity systems, pressure management systems, vapor managementsystems and selection of particularized equipment and components.

From the foregoing description, one of ordinary skill in the art caneasily ascertain the essential characteristics of this disclosure, andwithout departing from the spirit and scope thereof: can make variouschanges and modifications to adapt the disclosure to various usages andconditions. The embodiments described hereinabove are meant to beillustrative only and should not be taken as limiting of the scope ofthe disclosure.

Heating Up and Cooling Down of the Tank in Response to a Shock Event

In one example embodiment, an immersion cooling system or a vessel caninclude a tank, a computing device, a robot, an absorption unit, abellows and a management system. The tank can be a pressure controlledtank maintained at the atmospheric pressure (or within a range thereof).The tank can include a bath area and a sump area, and the computingdevice can be immersed in a dielectric fluid in the bath area of thetank. The computing device can be connected to a network and performvarious processing tasks while immersed in the dielectric fluid. Thetank can include a lid for accessing the bath area, the computing deviceand the sump area. The tank can be fluidly coupled to the bellows andthe absorption unit, and a plurality of valves can selectively connector disconnect the tank to and from the bellows and the absorption unitso that dielectric vapor can transfer to the bellows and/or absorptionunit, or vice versa. The robot can be a gantry robot, which can lift thecomputing device from the tank of the vessel when the lid of the tank isopen. The robot can place the lifted computing device in a magazineprovided for storage of computing devices. The robot can also lift acomputing device from the magazine and place it in the place of thelifted computing device from the tank.

In one example embodiment, the tank can include a heating element, e.g.,a plurality of heating rods some of which are at least partly immersedin the dielectric fluid. The tank can include a plurality of sensors,e.g., temperature sensors, pressure sensors or sensors which provideoperating data relating to the computing device (e.g., current, voltage,workload, etc.). The temperature sensors can be located inside the bathor in an area above the bath. Using data received from the sensors, themanagement system of the vessel can operate the heating element toregulate or control the temperature or temperature fluctuations of thedielectric fluid (and/or pressure or pressure fluctuations of thedielectric vapor) in the tank. FIG. 18 shows an example of a heatingelement 1000 for an immersion cooling system according to an exampleembodiment. The heating element 1000 can include a plurality of heatingrods 1010. Each heating rod can include a plurality of wires 1011, whichcan be connected to a power supply of the tank. A controller of the tankcan regulate the heating element 1000 to heat up the bath area of thetank, e.g., during various operations of the tank. In this exampleembodiment, the heating element 1000 can be mounted in the bath of thetank and fully immersed in the dielectric fluid.

In one example embodiment, the heating element is separate from thecomputing device and the heating element does not process data. Theheating element may be dedicated to only generating heat and no otherfunction. The heating element may be readily controlled particularlyduring an operation of the tank (e.g., startup operation), a change ofcomponents, or other times when control is necessary. The heat generatedby the heating element may be adjustable upon evaluating data indicatingthe size of the bellows and other aspects of the system, e.g., pressureor temperature.

In particular, rapid changes in power consumption or workload of thecomputing device (e.g., caused by the end user activity or lack thereof)can result in rapid changes to the amount of heat generated by thecomputing device within the vessel. This in turn can cause a rapidtemperature change within the tank or the bath, which can result in asudden change in the pressure in the tank (because in a closed insulatedsystem, the pressure and temperature are directly related, i.e.,PV=nRT). These pressure fluctuations can damage the vessel and introducecontaminant gasses (e.g., air) or particulates (e.g., dust) to the tank.These pressure fluctuations can also cause leakage of the dielectricfluid from the tank. To counter the effects of these pressurefluctuations, it is possible to use the bellows or the absorption unitto remove excessive vapor from the tank or introduce vapor into the tankwhen the pressure drops. However, by using a heating element, thecapacity of the bellows and absorption unit can be reduced, thereby amore space efficient vessel can be designed. If the heating element isnot used, in the event of an excessive increase in the pressure, thebellows can burst.

The heating element can allow for modulation in change of temperaturewithin the tank or the bath, thereby facilitating a controlledtransition between the various operating load conditions that may beexperienced by the computing device during its operation. For example,in the event of a rapid decrease in operational workload of thecomputing device, the heat generated by the computing device can droprapidly. This can cause a sudden decrease in the internal pressure ofthe tank. The heat element can add heat to the tank to allow for acontrolled decrease in the temperature of the dielectric fluid, e.g.,during a shutdown process. In other words, the heating element canbalance the pressure and temperature of the tank in the event of asudden change in the workload of the computing device, i.e., a shockevent. As such, the vessel requires a much smaller bellows andabsorption unit to maintain the atmospheric pressure of the tank.

In one example embodiment, the management system of the vessel candecide how much heat to add to the tank in response to a shock event,e.g., an increase or decrease in the internal pressure or temperature ofthe tank. In one example embodiment, the rate of the drop (or increase)in temperature or pressure can determine how much heat to add to thetank. For example, in the event that the tank's dielectric fluidtemperature level decreases more than a certain number of degrees, overa certain number of minutes, the management system can activate theheating element to add a certain amount of heat to the tank (e.g., tomaintain the temperature and pressure of the system). This added heatcan cause the fall in temperature to stop or the rate at which thetemperature is falling to decrease. The management system can stop theheating element from adding heat to the system when the tank is in asteady state, e.g., the rate of fall in pressure or temperature is belowa threshold value. In another example embodiment, the actual temperatureof the dielectric fluid within the tank when the increase or decrease ofthe workload of the computing device initiated can determine how muchheat to add to the tank.

In one example embodiment, the management system can activate theheating element when a shock event is detected, e.g., before, during orafter a startup operation, a boost operation, a slowdown operation, or ashutdown operation. The management system can detect the mode ofoperation of the vessel (e.g., startup or shutdown) by receiving sensordata (e.g., from the temperature or pressure sensors in the tank) ordata from the computing device (e.g., current, voltage, temperature,workload, data transfer, etc.). The heating element can moderate orregulate the change in temperature or pressure within the tank tominimize the deviation of the pressure from an atmospheric pressure.Otherwise, without operation of the heating element according to thetechniques disclosed herein, the vessel will have to either absorb orstore the excessive gas generated as a result of a rapid heat up of thecomputing device or the vessel will have to desorb or supply gas tonegate a drop in pressure as a result of a rapid drop in heat generationof the computing device.

During a startup operation, a temperature of the tank, e.g., temperatureof the dielectric fluid in the bath, is below a threshold value when thecomputing device starts operating. The startup operation can occur,e.g., when the vessel is just turned on and the tank is cold. Becausethe computing device can heat up quickly, the computing device cangenerate a lot of vapor if the dielectric fluid is cold. Therefore,before, during or after the startup operation, the management system canactivate the heating element to heat up the dielectric fluid, therebyincreasing the temperature of the dielectric fluid in a controlledmanner and minimizing the vapor generation by the computing device. Forexample, the heating element can slowly increase the temperature of thedielectric fluid to a threshold temperature before the computing deviceis turned on. Otherwise, the vessel will have to accommodate anexcessive quantity of vapor to maintain the tank at atmosphericpressure, which can require a large capacity at the bellows and theabsorption unit.

During a boost operation, a temperature of the tank, e.g., temperatureof the dielectric fluid in the bath, can increase faster than athreshold rate (e.g., when the temperature of the tank is below athreshold value). The boost operation can occur, e.g., when thecomputing device is operational and the workload of the computing deviceincreases significantly, e.g., due to an increase in consumer demand. Asudden increase in the workload of the computing device can increase theamount of heat generated by the computing device, thereby increasing theamount of vapor generated by the computing device. Therefore, before,during or after the boost operation, the management system can activatethe heating element to heat up the dielectric fluid, thereby increasingthe temperature of the dielectric fluid in a controlled manner andminimizing the vapor generation by the computing device. Otherwise, thevessel will have to accommodate an excessive quantity of vapor tomaintain the tank at atmospheric pressure, which can require a largecapacity for storage or absorption at the bellows and the absorptionunit.

During a slowdown operation, a temperature of the tank, e.g.,temperature of the dielectric fluid in the bath, can decrease fasterthan a threshold rate (e.g., when the temperature of the tank is above athreshold value). The slowdown operation can occur, e.g., when thecomputing device is operational and the workload of the computing devicedecreases significantly, e.g., due to a drop in consumer demand. Asudden decrease in the workload of the computing device can decrease theamount of heat generated by the computing device, thereby suddenlydecreasing the pressure in the tank. Therefore, before, during or afterthe slowdown operation, the management system can activate the heatingelement to heat up the dielectric fluid, thereby decreasing thetemperature of the dielectric fluid in a controlled manner andminimizing the pressure drop in the tank. Otherwise, the vessel willhave to generate a large quantity of vapor to maintain the tank atatmospheric pressure, which can require a large storage or desorptioncapacity at the bellows and the absorption unit.

During a shutdown operation (or a controlled shutdown process), thevessel is ordered to turn off while a temperature of the tank, e.g.,temperature of the dielectric fluid in the bath, is above a thresholdvalue. Because the computing device can stop generating heat suddenly,the pressure in the tank can drop rapidly. Therefore, before, during orafter the shutdown operation, the management system can activate theheating element to heat up the dielectric fluid, thereby decreasing thetemperature of the dielectric fluid in a controlled manner andminimizing the pressure drop. For example, the heating element canslowly heat up the dielectric fluid so that the temperature of thedielectric fluid slowly drops when the computing device is turned off.Otherwise, the vessel will have to generate a large quantity of vapor tomaintain the tank at atmospheric pressure, which can require a largestorage or desorption capacity at the bellows and the absorption unit.

In one example embodiment, as the vessel responds to a shock event, themanagement system (or another system) can add vapor or fluid to the tankor remove vapor or fluid from the tank to maintain the pressure of thetank at a pressure close to an atmospheric pressure. For example, as thetemperature of the tank increases, vapor or fluid can be removed fromthe tank and as the temperature of the tank decreases, vapor or fluidcan be added to the tank.

The vessel can use various mechanisms for adding vapor or fluid to thetank or removing vapor or fluid from the tank. In one exampleembodiment, the vessel can use a bellows as the mechanism for addingvapor to the tank or removing vapor from the tank. In another exampleembodiment, the vessel can use an absorption/desorption unit(hereinafter “absorption unit”) for adding vapor to the tank or removingvapor from the tank. Yet in another example embodiment, the vessel canuse a pressurized container for adding vapor to the tank or removingvapor from the tank. Yet in another example embodiment, the vessel canuse a combination of the above-named mechanisms for adding vapor orfluid to the tank or removing vapor or fluid from the tank. Yet inanother example embodiment, the vessel can use a combination of theheating element and one or more of the above-named mechanisms tomaintain the pressure of the tank.

For example, during a startup operation, the management system can use acombination of the heating element and the bellows to maintain thepressure of the tank. In one example, before the computing device isturned on, the management system can activate the heating element toheat up the dielectric fluid. At some point (e.g., before, after orduring the heating), the management system can open a valve connectingthe bellows to the tank, thereby facilitating a transfer of thedielectric vapor to the bellows. This transfer of the dielectric vaporto the bellows can prevent an uncontrolled increase in the pressure ofthe tank, thereby allowing for the temperature of the dielectric fluidto increase while the pressure of the tank can be maintained (e.g.,within a tolerance range).

Similarly, during a startup operation, the management system can use acombination of the heating element and the absorption unit to maintainthe pressure of the tank. At some point (e.g., before, after or duringthe heating), the management system can open a valve connecting theabsorption unit to the tank, thereby facilitating a transfer of thedielectric vapor to the absorption unit, which can absorb or maintainthe dielectric vapor in the absorption unit, e.g., a carbon bed.Similarly, during a startup operation, the management system can use acombination of the heating element and the pressurized container tomaintain the pressure of the tank. At some point (e.g., before, after orduring the heating), the management system can open a valve connecting apump and the pressurized container to the tank, thereby facilitating atransfer of the dielectric vapor to the pressurized container using thepump. The pressurized container can store the dielectric vapor.

As another example, during a shutdown operation, the management systemcan use a combination of the heating element and the bellows to maintainthe pressure of the tank. In one example, after the computing device isturned off, the management system can activate the heating element toheat up the dielectric fluid. At some point (e.g., before, after orduring the heating), the management system can open a valve connectingthe bellows to the tank, thereby facilitating a transfer of thedielectric vapor to the tank. This transfer of the dielectric vapor tothe tank can prevent an uncontrolled decrease in the pressure of thetank, thereby allowing for the temperature of the dielectric fluid todecrease while the pressure of the tank can be maintained (e.g., withina tolerance range).

Similarly, during a shutdown operation, the management system can use acombination of the heating element and the absorption unit to maintainthe pressure of the tank. At some point (e.g., before, after or duringthe heating), the management system can open a valve connecting theabsorption unit to the tank, thereby facilitating a transfer of thedielectric vapor to the tank. In the case of a carbon bed as theabsorption unit, the management system can activate the carbon bed torelease the trapped or absorbed dielectric molecules. The managementsystem can activate the carbon bed by, e.g., transmitting a signal to aswitch to turn on a heating device within the carbon bed. In oneexample, as the pressure of the tank drops, the carbon bed can be heatedup to release dielectric vapor and minimize the drop in pressure.

Similarly, during a shutdown operation, the management system can use acombination of the heating element and the pressurized container tomaintain the pressure of the tank. At some point (e.g., before, after orduring the heating), the management system can open a valve connectingthe pressurized container to the tank, thereby facilitating a transferof the dielectric vapor to the tank.

In one example embodiment, there can be a tradeoff between using thebellows and the absorption unit. The bellows is a passive device, butthe absorption unit is an active device. A bellows-based system can bemore power efficient than an absorption-unit-based system because thebellows does not require active heating. However, the bellows takes morespace than the absorption unit and the absorption-unit-based systemprovides a greater degree of control and functionality. Some of thedesign constraints in this regard can be efficiency, control and space.

In one example embodiment, the vessel can experience an uncontrolledshutdown. For example, the vessel can experience an uncontrolledshutdown due to loss of power. In this example embodiment, an emergencyshutdown process can be implemented to address possible pressurefluctuations in the tank. For example, the vessel may have a backup oruninterruptible power supply (“UPS”) that can supply power to the vesseland its management system (or another system). If the management systemreceives a signal from the sensors indicating that, as a result of thepower loss and cooling down of the system, the pressure inside the tankis dropping below an acceptable threshold, the management system caninstruct a bypass valve to open. The bypass valve can connect the tankto the environment outside of the tank. The bypass valve can introduceair to the tank, and therefore, normalize the pressure inside the tank(so that the tank or the bellows would not collapse). Subsequently,during the startup operation, the vessel can purge the air introducedinside the tank.

In one example embodiment, the management system (or another system) canuse a table, matrix or map (“map”) for determining how to respond to ashock event. In one example embodiment, the map can display a change intemperature as an input and display an output as to how much heat to addto the tank in response to the change in temperature. In one exampleembodiment, the map can include data relating to a vapor temperature, atank pressure, a fluid level in the tank or sump area, a fluid pressurein a pump or filter, a differential pressure, a humidity level, and analumina condition as input. In response to these inputs, the map canprovide outputs such as condenser, heating element, pump, bellows valve,carbon intake valve, carbon exhaust valve, and computing deviceoperating parameters. The map can define various states for theoperation of the vessel. The management system can receive various datafrom sensors provided throughout the vessel. Using the map, themanagement system can convert the data to operating parameters for thedevices on the vessel, e.g., the bellows, absorption unit, valves,heating element, pump, condenser, and computing device.

In one example embodiment, the vessel can be operated at a temperaturenear the boiling point of the dielectric fluid and a pressure near theatmospheric pressure. However, one of ordinary skill in the artrecognizes that based on the optimal operating temperature for operatingthe computing device, the vessel can be operated at other temperatureand pressure ranges. In one example embodiment, the optimal operatingtemperature of the system is about 137° F.±8°. In one exampleembodiment, the optimal operating pressure of the system is about theatmospheric pressure (e.g., 101,325 Pa)±5,000 Pa. In this exampleembodiment, during a shock event, the management system will try tomaintain the temperature and pressure of the vessel within these ranges.

Although in some example embodiments of this disclosure the managementsystem is designated as the system programmed to carry out various tasksin a shock event, one of ordinary skill in the art recognizes that othersystems disclosed in this disclosure can be programmed to perform thesetasks.

In one example embodiment, the vessel can operate under three modes ofoperation. In a first mode of operation, the tank can operate atatmospheric pressure. In a second mode of operation, the tank canoperate at pressure ranges that significantly deviate from theatmospheric pressure. In a third mode of operation, the vessel cansometimes operate at atmospheric pressure and sometimes operate atpressure ranges that significantly deviate from the atmosphericpressure. The third mode of operation can be a hybrid of the first modeand the second mode. In one example embodiment, the management systemcan determine the mode of operation of the vessel. For example, themanagement system can operate the vessel based on rules defined for themanagement system, e.g., pressurize the vessel every morning at 5 AM andreturn to atmospheric pressure at night; pressurize the vessel duringpeak of workload as determined by sensor data. As another example, themanagement system can use a machine learning algorithm to predict themode of operation for the vessel. For example, the machine learningalgorithm can use sensor data as well as exogenous data, e.g., weathercondition, calendar data, usage data, etc., to predict which mode ofoperation is more efficient under the circumstances. A user of thesystem can provide labeled data to the management system, which canextrapolate the data to create a model for predicting the mode ofoperation.

In one example embodiment, the management system can perform a specificroutine before a lid of the tank can be opened. For example, if thevessel is provided with an instruction to open a lid of the tank, thecondensing system can cool the system for a period of time before themanagement system allows for the lid to open. The condensing system canminimize the vapor in the tank so that when the lid is opened, minimaldielectric vapor is lost to environment.

In one example embodiment, the immersion cooling system can be a modularsystem. For example, each group of components of the system can bemounted on a separate skid, e.g., a condensing skid, a heating skid, abellows skit, an absorption unit skid, etc. These skids can be movableand deployed for various applications.

Dielectric Fluid Circulation and Filtration

In one example embodiment, the vessel can include a pump for circulatingthe dielectric fluid through the tank. For example, the tank can includea sump area and a bath area. The bath area can hold the computing deviceimmersed in the dielectric fluid. The sump area can be next to the batharea, or the sump area and can be in fluid communication with the batharea. For example, the sump area can receive an overflow of thedielectric fluid from the bath area, e.g., the dielectric fluid can flowover a wall of the bath area adjacent to the sump area. The pump candraw the dielectric fluid from the sump area and pass the fluid througha filter. After the filter, the dielectric fluid can return to the batharea. The vessel can include various pipes that couple the sump area,the pump, the filter and the bath area.

In one example embodiment, the vessel is provided with an amount ofdielectric fluid such that the bath area is full of dielectric fluid andthere is an overflow of dielectric fluid in the sump area. A full batharea ensures that the computing device is fully immersed in thedielectric fluid. The pump can draw the dielectric fluid from the sumparea and pass it to the bath area, e.g., through the filter. Becausethere is more dielectric fluid in the tank than the bath area's capacityto hold fluid, when the pump runs, the bath area is always full(particularly when the pump operates). However, depending on the tank'stemperature, the level of the dielectric fluid in the sump area canchange because dielectric fluid can evaporate from the bath area anddielectric fluid from the sump area can replace the evaporated fluid inthe tank.

In one example embodiment, the tank can be the shape of a rectangularbox. The dielectric fluid can flow over the top of one of the shortersides into the sump area, which is adjacent to the shorter side. Thepump can draw the dielectric fluid and return or reintroduce it to alocation in the tank that can cause minimum disruption or turbulence tothe fluid in the bath as disruption or turbulence can cause cavitationin the fluid. In particular, the longer the distance between theoverflow area to the reintroduction point, the less the turbulenceassociated with the reintroduction of the fluid into the tank. Forexample, if the dielectric fluid overflows from the top of a first sideof the bath, the pump can return the dielectric fluid to the bottom of aside opposite to the first side. The pump can return the dielectricfluid to a corner of the bottom side, which minimizes disruption orturbulence to the fluid in the bath.

In one example embodiment, the vessel can include two pumps. Each pumpcan independently draw fluid from the sump area and pass it to the batharea. Providing the vessel with two separate and independent pumps canenhance the service life of the vessel. Additionally, if one of thepumps fails for any reason, the vessel can continue its operationswithout disruption until the failed pump is exchanged.

In one example embodiment, the vessel can include a filter. The filtercan include one or more cores. Each core can filter the dielectric fluidfor a different type(s) of contaminant, particle, substance, diluent orsolute. In one example embodiment, the cores can be chosen based on theproperties of the dielectric fluid and the contaminants likely to beintroduced into the dielectric fluid. For example, the contaminants caninclude solder and resin, which are used during the manufacturingprocess of electronic boards used in the computing device. Thedielectric fluid can act as a cleaning agent for resin, solder, dust,dirt or anything else in the system. Solder and resin (or othersubstances) can wash off from these electronic boards after they areimmersed in the dielectric fluid. The filter can remove solder and resin(or other substances) from the dielectric fluid. If these substances arenot removed from the dielectric fluid, when the dielectric fluidvaporizes, these substances will deposit as a layer on the heatgenerating components of the computing device, e.g., processors. As aresult, the layer thermally isolates or insulates the heat generatingcomponents from the dielectric fluid, thereby reducing the efficiency ofheat transfer from these components to the dielectric fluid. Thus, thecomponents can heat up and break down more frequently.

In one example embodiment, the filter can include two cores, one coreincluding activated carbon (charcoal) and one core including activatedaluminum. For example, the ratio of activated carbon to activatedaluminum can be 3 to 1. As another example, the filter can include fourcores, three cores including activated carbon and one core includingactivated aluminum.

In one example embodiment, the filter can include a stripe for testingacidity of the dielectric fluid. The stripe can be a PH indicator,litmus paper or other indicator. In some instances, the dielectric fluidcan become acidic after interaction with certain components of the tank.The stripe can be in contact with the dielectric fluid and change colorif the dielectric fluid becomes acidic. The filter can include a colordetection sensor, which can detect a change in color in the stripe andtransmit a signal to the management system (or another system) if achange in color of the stripe is detected. In one example embodiment,the stripe can be disposed in a container or chamber including a glassshield. As such, a change in color of the stripe can be visible outsideof the container. A camera can be disposed within a vicinity of thecontainer. The camera can take a photo of the stripe (behind the glassshield) and transmit the photo to the management system. If themanagement system (or a user of the system) detects a change in thecolor of the stripe (using data provided by the camera or the colorsensor), the management system can trigger remedial action, e.g., notifya maintenance system or shutdown the system.

In one example embodiment, the camera can be a pan-tilt-zoom camera. Afilter lid can be mounted on top of the sump area. The filter lid can beinstalled next to other lids which provide access to the bath area. Thefilter lid can include the filter and the camera can be installed on thefilter lid. In one embodiment, the camera can be installed immediatelybelow the filter lid. As such, when the camera rotates, the camera cantake images of the stripe, the sump area (the area below the camera) andthe area over the bath area.

FIGS. 19A-B show a filter including three cores according to an exampleembodiment. As shown in FIG. 19A, the filter can include a lid 1050which can be mounted on the tank, e.g., next to other lids which provideaccess to the computing device installed within the tank. Each core ofthe filter can be connected to the lid 1050. The lid 1050 can includethree caps 1060, each cap providing access to one of the cores. FIG. 19Bshows a structure 1070 mounted to the lid 1050. The structure 1070 cansupport various filter cores and other components, e.g., filter core1071, camera 1072 and electromechanical valve 1073. On the other side ofthe structure 1070, there can be two other filter cores (not show inFIG. 19B).

In this example filter, there is a camera and two color sensors attachedto the lid. The camera and the color sensors can obtain data relating tothe acidity of the dielectric fluid (based on the color of the stripe)and convey the data to the management system.

In one example embodiment, the filter can be mounted on a chassisremovable by a robot. The chassis can include a connection interface fordetachably connecting the chassis (and the filter provided therein) tovarious pipes provided in the tank. As such, when the management systemdetermines that the filter needs to be replaced, the robot can lift thechassis out of the tank, and place the filter in the magazine.

In one example embodiment, the management system can notify a user whenthe filter needs to be serviced or replaced. For example, the managementsystem can include a timer or counter which is activated when the filteris installed on the vessel. If the management system determines that thefilter has been in operation for more than a threshold time, themanagement system can transmit a notification to the user (or otherentity). As another example, the management system can activate thetimer or the counter only when the vessel is in operation, the pump isactive or the dielectric fluid passes through the filter (as determinedby a fluid sensor in the filter). If the management system determinesthat the filter has been in operation for more than a threshold time,the management system can transmit a notification to the user. As yetanother example, the management system can determine a pressuredifferential for the filter, and the management system can notify theuser to service or replace the filter if the pressure differentialexceeds a threshold pressure. In particular, the filter can include aninput pipe and an output pipe, and there can be a pressure sensor on theinput pipe and a pressure sensor on the output pipe. Each pressuresensor can transmit a pressure reading to the management system. If thepressure differential between the readings of the pressure sensorsexceeds a threshold pressure, the management sensor can determine thatthe filter is clogged. Therefore, the management system can notify theuser to service or exchange the filter. As yet another example, thefilter can include a sensor which indicates that flow rate for thefilter. The management system can use the flow rate to determine if thefilter needs service. As yet another example, the management system canuse a machine learning model to determine when to replace the filter.The model can receive training data from a central server indicatingoperational data for filters for a plurality of vessels connected to theserver.

In one example embodiment, the sump area and/or bath area can includeone or more fluid level sensors. During startup or a rapid increase inthe workload, as the dielectric fluid vaporizes in the bath area, thefluid level drops in the sump area. However, because the pump circulatesthe dielectric fluid, the fluid level in the bath area remains the same,i.e., the computing device remains immersed. Conversely, during shutdownor a rapid decline in the workload, the fluid level can decrease in thesump area.

The fluid level sensors can provide data to the management systemregarding the fluid level in the sump area and the bath area. If afterthe startup of the vessel (or while the vessel is operating in a steadystate) the fluid level in the sump area decreases, there can be a leakin the tank. Similarly, if at some point the fluid level in the batharea drops, there can be a problem with the fluid circulation system,e.g., the pumps have failed. As such, the management system cancontinuously monitor the fluid level data provided by fluid levelsensors and notify the user if there is an unexpected drop in fluidlevel in the sump area or the bath area.

In one example embodiment, the pump can draw fluid from the sump area(or bath area) and provide the fluid to a drain valve connected to thetank body. When the valve is open, the pump can drain the sump area (orthe bath area) or provide a sample to a user of the vessel. The samplecan be provided to a lab for testing. In one example embodiment, theuser can instruct the vessel, using the management system, to drain thetank. In response, the management system can open the drain valve andthe pump can direct the fluid from the sump area (or even the bath area,e.g., when a connection is provided to the bath area) to the drainvalve. For example, there can be a valve connection between the batharea and the sump area, and in the event a draining instruction isreceived, the valve connection can connect the bath area to the sumparea so that the bath area drains the dielectric fluid into the sumparea, and the pump drains the sump area. In one example embodiment, thepump can draw the dielectric fluid directly from the tank area.

The Robotic System

In one example embodiment, the vessel can include a robotic system,e.g., a gantry robot configured to lift the computing device from thebath area of the tank or a magazine placed near the tank. The gantryrobot can also place the computing device in the bath area or themagazine.

The gantry robot (or the robot) can include a series of linearactuators. For example, the robot can include an actuator for movementin each of a plurality of directions, e.g., horizontal and vertical. Themanagement system (or another system) can control how much or how fasteach one of these actuators move. In one example embodiment, theactuators can be configured to move on one or more tracks.Actuator-based or track-based systems can lose their precision over time(e.g., due to drifting or wear and tear). As such, in this exampleembodiment, in order to detect the exact relative position of the robot,the tank (or various components thereof) can include one or morecalibration zones or flags. For example, one or more of the keycomponents or locations of the vessel with which the robot interacts,e.g., the magazine, the first server rack, the second server rack, orthe home position, can include a flag, which can be detected by therobot once the robot reaches the position of the key component orlocation. The flag can notify the robot about the exact location of therobot relative to the key component or location.

In one example embodiment, the flag can be a physical object, an RFIDtag, a color, an alphanumerical code, a QR code, etc. In one exampleembodiment, the sensor detecting the flag can be a motion sensor, anRFID detector, a camera, etc. In one example embodiment, the camera candetermine a distance between the robot and various objects and providefeedback to the robot regarding the distance. In one example embodiment,the camera can provide video data to the management system, and based onthe video data, the management system can determine the exact locationof the robot within the vessel. In one example embodiment, themanagement system can determine the location of the robot by, e.g.,scanning a QR code, counting the components in the tank, etc. In oneexample embodiment, the images from the camera can be used to determinethe proximity of the robot to an object or whether the robot hasproperly grabbed or placed a chassis. In one example embodiment, themanagement system can use an object recognition technique to determinethe location of the robot. In one example embodiment, the managementsystem can use an artificial intelligence technique to determine thelocation of the robot. The management system can use the objectrecognition technique or the artificial intelligence to calibrate therobot.

In one example embodiment, the vessel can include a home position, amagazine and two racks. The management system (or another system) caninstruct the robot to lift a computing device from, e.g., the secondrack. The robot can move from the home position to the magazine and thento the first and second racks. As the robot approaches each of theselocations or components, a sensor of the robot can detect the associatedflag for the location or component. The benefit of the flag system isthat the robot can still detect the robot's position relative to a keycomponent or location even if other components or locations have beenremoved from the vessel. This is because the flags are always located atthe same position relative to each key components or locations the flagsare associated with. For example, even if the first rack is removed fromthe vessel, the robot can find the flag for the second rack, calibratethe robot's position relative to the second rack and remove thecomputing device from the second rack. Similarly, even if the secondrack is slightly moved from its position in the vessel, the robot canfind the flag for the second rack, calibrate the robot's positionrelative to the second rack and remove the computing device from thesecond rack.

In one example embodiment, the gantry robot can receive instructions toremove or replace various components of the vessel, e.g., computingdevice, filter, etc. In one example embodiment, the instructions can beprovided by the management system (or another system). The managementsystem can provide the instructions in response to a determination bythe management system (or another system), the user of the vessel or asystem external to the vessel. For example, the management system canreceive and monitor various data points relating to operation of thecomputing device, e.g., voltage levels, temperatures and other operatingproperties. If the computing device exceeds the thresholds determined orpredetermined for the computing device, the management system isprogrammed to instruct the robot to replace the computing device.

As another example, the user of the vessel can direct the managementsystem to provide the instruction to the robot to remove the computingdevice. As yet another example, the management system an include anapplication programming interface (API) for receiving instructions froma system external to the vessel. For example, the vessel can be incommunication with a top-level orchestration and management platformwhich can instruct the management system (through the API) to remove thecomputing device from the tank.

In one example embodiment, the robot can lift a computing device fromthe tank or the magazine. In this example embodiment, the computingdevice can be located in a chassis including a connection plate. Therobot can include a guide pin and a linkage mechanism which caninterface with the connection plate. The robot can also include one ormore load cells measuring the positive or negative force or pressureexerted on the robot.

The robot can start at its home position and move toward the tank (orthe rack including the computing device). At the tank, the robot candetect the flag associated with the tank, which can inform the robotthat it is at the tank. Then, the robot can move a predetermineddistance from the flag so that the robot is located exactly (orsubstantially) over the computing device. Once the robot is on top ofthe computing device, the robot can rapidly drop from its top mostposition to a position a few inches away from the computing device (or aposition equal to or longer than the length of the guide pin, e.g., 50%longer than the guide pin). At that point (i.e., a few inches away fromthe computing device), the robot can approach the computing deviceslower so that the guide pin of the robot makes an initial contact withthe connection plate. Once the robot makes the initial contact, therobot will continue moving down with the same slow speed until the robotpresses the connection plate (of the chassis) by more than a thresholdpressure. At that point, the linkage mechanism of the robot can activate(e.g., the fingers can open) to interconnect the robot to the computingdevice. The linkage mechanism can be an armature-based linkage mechanismincluding a plurality of fingers. Once the linkage mechanism closes, therobot can provide feedback to the management system, i.e., that thelinkage system is closed. The management system can instruct the robotto lift the computing device. The robot can slowly pull up the chassisfor a few inches, to make sure that it has a good grip on the chassis.Then, the robot can rapidly move up to its top most position. At thatpoint, the robot can move to any position instructed by the managementsystem, e.g., the magazine or another rack.

In one example embodiment, the robot can place a computing device in thetank or the magazine. For example, while holding a chassis, the robotcan move over a slot of the tank (or one of its racks) to drop or placethe chassis inside the tank (or the magazine). Once the robot is overthe tank, the robot begins to move down rapidly, until it gets to a fewinches above the first alignment point (or mating point) between thechassis and the tank (or the rack). The design of the chassis and thetank can determine the distance from above the tank at which the robotshould slow down. In particular, the robot can slow down one or twoinches above the alignment point (where the guide rails of the chassiscontact the grooves of the rack). The robot can move slowly toward thetank so that grooves of the rack can move in the guide rails of thechassis. The management system can receive and monitor data from theload cell and other sensor to ensure that the chassis is not misaligned.For example, an excessive amount of force feedback on a load cell canindicate a misalignment between the grooves and the guide rails. Themanagement system can abort the drop operation if a misalignment isdetected.

In one example embodiment, in addition to the grooves and the guiderails, the chassis and the rack can include additional alignmentmechanisms. For example, after the initial mating between the chassisand the rack using the groove and the guide rails, a guide pin mechanismcan be provided on the rack and the chassis which can further align therack and the chassis. The guide pin mechanism can include a pin on therack and a mating hole on the chassis. After the initial mating, therobot can again move down rapidly until it reaches the second alignmentmechanism (or a few inches thereof, e.g., two inches more than the sizeof the guide pin). Here, the second alignment mechanism can be the guidepin mechanism. The robot slowly moves down so that the pin on the rackcan connect with the mating hole on the chassis. The robot continues tomove down slowly until the load cell provides a feedback indicating thatthe chassis has been inserted, e.g., the load cell detects a positivepressure. At this point, the linkage mechanism can deactivate, and therobot can move up (slowly for a few inches to ensure proper placementand then rapidly) and back to its home position.

In one example embodiment, the chassis or the rack can include apresence detection pin. When the presence detection pin mates with thecorresponding receiver, the management system can receive a signal fromthe receiver. The signal can indicate that the chassis is properlyplaced in the position. In this example embodiment, the robot candeactivate the linkage mechanism only after the receiver provides thesignal to the management system.

During the lift or drop operations, the management system can receiveand monitor the data received from the load cells as well as othersensors (e.g., a motion sensor, a tile sensor, a rotation sensor, anaccelerometer, etc.). This data can ensure that the chassis is not stuckor misaligned, or that the robot has a good grip of the chassis. If themanagement system determines that the chassis is somehow stuck ormisaligned, or that the robot is tilting or rotating (e.g., because ofthe robot's poor connection with the chassis), the management system canabort the lift or drop operation.

FIGS. 20A-B show an example robotic system 1100. FIG. 20A shows therobotic system 1100, which can be a gantry robot including a plate 1110.The gantry robot can move within the tank and lift the computing deviceusing the plate 1110. FIG. 20B shows the plate 1110 which can include alinkage mechanism 1111 and a guide pin 1112. The linkage mechanism 1111can include a plurality of fingers 1113, which are mechanically coupledto one or more armatures. Once the linkage mechanism is placed in aconnection plate of a chassis, the armature can activate and move thefingers to hold the connection plate.

FIGS. 21A-B show an example guide pin mechanism between a chassis 1150and a rack 1160. In this example embodiment, the rack 1160 can includetwo guide pins 1161 (for each chassis 1150) and the chassis 1150 caninclude two mating holes 1151 configured to receive the guide pins 1161.When a robot drops the chassis 1150 over a rack 1160, the guide pinmechanism ensures proper electrical connection between the chassis 1150and the rack 1160.

In one example embodiment, the robot is a robotic arm. The robotic armcan move on a rail provided on a side of the tank. In one exampleembodiment, each chassis can be pulled up using a piston and connect toa channel located on top of the piston. The channel can deliver thechassis to the magazine, e.g., using a rail system.

In one example embodiment, the robot can include a calibration system,which can include a plurality of sensors. The calibration system candetermine if the robot is exceeding its normal range of operation. Forexample, a tilt sensor can inform the robot if the robot is not balancedor tilted. As another example, a load cell can provide a signal to therobot if the robot is not moving freely, e.g., hits an object.

In one example embodiment, the robot can use an artificial intelligenceor machine learning technique to provide for hot swapping or as afailsafe mechanism.

In one example embodiment, the vessel can include a plurality ofcameras. In this example, one camera can be mounted on the robot andanother one can be mounted on a vessel wall. The cameras can be mountedin a way that the user always has visibility into the moving componentsof the vessel. The vessel can also include a user interface displayed ona display device, e.g., monitor. When the robot is lifting or dropping achassis, the user interface can display video feed from the cameras.This way, the user can take action if anything goes wrong with therobot's operation.

In one example embodiment, the robotic system can be a vision-basedsystem that is tied to active control. The active control allows areference points to be sent back through logic that will determineproximity with a proximity switching. In one example embodiment, therobotic system can be an AI robotic system. In one example embodiment,the robotic system can be an auto correcting system. In one embodiment,the robotic system can be a logic controlled active loop based system,that is preprogramed and can calibrate based on distances from itsresting state to its active state.

The Absorption/Desorption Unit

In one example embodiment, the absorption unit can be a carbon-bed-basedsystem. The absorption unit can be a round circular drum. Inside theabsorption unit there can be an aluminum framework which allows forinclusion of copper ribbon heating elements that are spread throughoutthe framework. The height and radius of the absorption unit can bedesigned based on the size of the vessel and the volume of fluid in thetank.

The absorption unit can be sealed and include activated carbon withinthe framework. The absorption unit can include an inlet and an outlet.In one embodiment, the absorption unit can include a cooling system,e.g., cool air can flow through the center of the absorption unitwithout making contact with the carbon. This system allows for coolingthe carbon through convection.

In one example embodiment, the activated carbon allows for absorption oradhesion of the dielectric vapor. When necessary to balance the tank(e.g., to create pressure or vacuum), the management system can connectthe absorption unit to the tank by opening a valve. The managementsystem can activate or initiate power to the copper heating ribbonelements, which can heat up the carbon. The carbon then releases thedielectric fluid molecules as vapor, which can return to the tank.

In one example embodiment, there can be pressure and temperature sensorsin the carbon bed to prevent a pressure or temperature over condition.

In one example embodiment, the absorption unit can include a controlloop bypass for pressure release (or pressurization) in emergencies.This is a safety feature of the vessel. The absorption unit has a valvewhich disconnects it from the tank. If the valve fails, there can be anover-pressurization condition. For example, the management system canopen the bypass valve if an outlet of the absorption unit is clogged. Ifthe bypass valve is opened, the dielectric vapor can go to theatmosphere, thereby preventing an explosion of the absorption unit. Ifthere was a vacuum condition within the tank, the valve can open toallow for air to come in the tank to prevent the tank from collapsing.

In one example embodiment, the vessel can include a plurality of safetybypass valves. For example, during a startup operation, the computingdevice can generate an excessive amount of vapor. The valve allowing thevapor to exit the tank to the absorption unit might fail. As such, therecan be an over pressurization condition in the tank which the bellowscannot handle. An emergency bypass valve can be opened to release someof the vapor into the atmosphere.

As another example, during the startup operation, an excessive amount ofvapor can enter the absorption unit. This can create anover-pressurization condition in the absorption unit. As such, thebypass valve of the absorption unit can be opened to release the vaporto the atmosphere.

In one example embodiment, in addition to pressure and temperature, themanagement system can receive and monitor data from the absorption unitrelating to electrical power of the absorption unit. The managementsystem ensures that current is running through the absorption unit. Themanagement system can shutdown the absorption unit if there is anovercurrent issue or if there is an over-pressurization condition.

Self-Alignment of the Chassis

In one example embodiment, the chassis can include a self-alignmentfeature. The self-alignment feature can include a plate which can bemovable (i.e., floats) relative to the chassis. There can be one or moreinput or output ports (or connectors) on the plate. The chassis (and theplate) can include a mating hole, which can receive a guiding pin toalign the plate for receiving the ports. In one example embodiment, theport can be an ExaMAX® connector.

In one example embodiment, the self-alignment feature can includeseveral alignment mechanisms. For example, as a first self-alignmentmechanism, the rack and the chassis can have grooves and guide rails. Asa second self-alignment mechanism, the rack can include a pin that istapered and rounded in the end. The pin can go into a catch hole in thechassis. As the pin is inserted in the mating hole (or catch hole), itmakes the final precision alignment between the connector on the chassisand the interface on the rack (i.e., the backplane). By the time theconnector is ready to pair with its mate, the alignment pin has causedthe floating pair to be perfectly matched up with the relativeorientation of the connector to which it is going to be inserted.

In one example embodiment, the connectors of the chassis and the rackcan include their own alignment mechanism, e.g., the pins can be part ofthe connectors.

In one example embodiment, the connectors can include multistagemechanisms for self-alignment, including a gross outer alignment catch,and then a finer inner alignment catch.

In one example embodiment, the plate can be on the backplane of therack.

FIG. 22 shows example connectors with self-alignment features. Theseconnectors can include guide pins and other guiding features to ensureproper connection between the pairs

An Exemplary Embodiment of Absorption/Desorption Unit

In one example embodiment, an immersion cooling system or a vessel caninclude a tank, a computing device, a robot, an absorption unit, and amanagement system. The tank can be a pressure controlled tank maintainedat the atmospheric pressure (or within a range thereof). The tank caninclude a bath area and a sump area, and the computing device can beimmersed in a dielectric fluid in the bath area of the tank. Thecomputing device can be connected to a network and perform variousprocessing tasks while immersed in the dielectric fluid. The tank caninclude a lid for accessing the bath area, the computing device and thesump area. The tank can be fluidly coupled to the absorption unit, and aplurality of valves can selectively connect or disconnect the tank toand from the absorption unit so that dielectric vapor can transfer tothe absorption unit, or vice versa. The robot can be a gantry robot,which can lift the computing device from the tank of the vessel when thelid of the tank is open.

In one example embodiment, the management system can activate theabsorption unit to cause a change in the pressure of the tank. In oneexample, the management system can activate the absorption unit tocontrol, moderate or regulate a change in the pressure within the tank,e.g., minimize the deviation of the pressure from an atmosphericpressure (or another predetermined pressure). For example, themanagement system can activate the absorption unit when a shock event isdetected, e.g., before, during or after a startup operation, a boostoperation, a slowdown operation, or a shutdown operation. The managementsystem can detect the mode of operation of the vessel (e.g., startup orshutdown) by receiving sensor data (e.g., from the temperature orpressure sensors in the tank) or data from the computing device (e.g.,current, voltage, temperature, workload, data transfer, etc.). In oneexample embodiment, the management system can respond to a shock eventby, e.g., instructing the absorption unit to add vapor or fluid to thetank or remove vapor or fluid from the tank to maintain the pressure ofthe tank at a pressure close to an atmospheric pressure (or anotherpredetermined pressure). For example, as the temperature of the tankincreases, vapor or fluid can be removed from the tank and as thetemperature of the tank decreases, vapor or fluid can be added to thetank.

The management system can use various mechanisms for adding vapor orfluid to the tank or removing vapor or fluid from the tank. In oneexample embodiment, the management system can use anabsorption/desorption unit (hereinafter the “absorption unit”) foradding vapor to the tank or removing vapor from the tank. In anotherexample embodiment, the management system can use another mechanism(s).Yet another example embodiment, the management system can use anothermechanism(s) in conjunction with the absorption unit.

The absorption unit may have different designs and advantageously do notrequire a bellows in some embodiments making for a smaller footprint andother advantages of the absorption unit. FIGS. 23-25 show an exampledesign for an absorption unit for an immersion cooling system. In thisexample embodiment, the absorption unit 100 can have a housing 131,which can be the shape of, e.g., a round circular drum (or any othershape). The housing 131 can have a top cover 132 which seals theabsorption unit against the surrounding environment. The absorption unit100 can include an air intake 111, an air output 113, a vapor intake 112and a vapor output 114. In one example embodiment, the absorption unit100 can also include an exhaust pipe 115, a pump 121 (or rotary fan121), an air filter 122 and a valve 123 (including a valve actuator124).

In one example embodiment, the valve 123 can be a four-way valveconnecting the air filter 122, the pump 121, the air output 113 and theexhaust pipe 115. FIG. 28 shows an example four-way valve according toan example embodiment. The valve 123 can include 2 or more modes ofoperation. In a first mode of operation of the valve 123, the air filter122 can be fluidly coupled to the pump 121. Additionally, the output 113can be fluidly coupled to the exhaust pipe 115. In a second mode ofoperation, the pump 121 can be fluidly coupled to the air output 113.Additionally, the air filter 122 and the exhaust pipe 115 can bedisconnected.

In one example, when the valve 123 is in the first mode of operation,the pump 121 can draw air from the filter 122. The pump 121 can push theair through a pipe to the air intake 111. By pushing the air to thehousing 131 through the air intake 111, the internal pressure of thehousing 131 can increase, and thus, air can flow out of the air output113, and back to the valve 123. Because the air output 113 is fluidlycoupled to the exhaust pipe 115, the air will exit from the exhaust pipe115 into the surrounding environment. In another example, when the valve123 is in the second mode of operation, the pump 121 can draw air fromthe air output 113. The pump 121 can push the air through a pipe to theair intake 111. By pushing the air to the housing 131 through the airintake 111, the internal pressure of the housing 131 can increase, andthus, air can flow out of the air output 113, and back to the valve 123.Because the air output 113 is fluidly coupled to the pump 121, the airwill recirculate within the housing 131. The recirculation mode can bebeneficial when, e.g., the temperature of the recirculated air is moreconsistent with the absorption unit's operation than the temperature ofthe incoming air from the air filter 122.

In one example embodiment, the absorption unit 100 can include a heater125. The heater 125 can be located on a pipe that connects the pump 121to the air intake 111. The heater 125 can heat up the air that entersthe housing 131. By heating the incoming air, the incoming air from theair intake 111 can heat up the absorption unit 100. Although the heater125 is described as being located on a pipe connected to the air intake111, one of ordinary skill in the art recognizes that the heater (orheaters) 125 can be located on other components, e.g., a pipe connectingthe air output 113 to the valve 123, the valve 123, a pipe connected tothe air filter 122, or within the housing 131. In one exampleembodiment, a cooling unit can be attached to the absorption unit 100.The cooling unit can chill the incoming air to or outgoing air from thehousing 131.

FIGS. 26-27 show an example design for elements inside the absorptionunit for an immersion cooling system. In one example embodiment, thehousing 131 can include a plurality of tubular elements 410. One ofordinary skill in the art recognizes that the elements 410 can bedesigned to be tubular or any other shape. The tubular elements 410 canform two isolated channels for transfer of fluids. The first channel canrun inside the tubular elements 410 and the second channel can runoutside the tubular elements 410 (but within the housing 131). In oneexample embodiment, the first channel can be connected to the vaporintake 112 and the vapor output 114. Accordingly, vapor flow 530 canenter into the tubular elements 410 from the vapor intake 112 and exitthe housing 131 from the vapor output 114. The second channel can beconnected to the air intake 111 and air output 113. Accordingly, airflow 520 can enter the housing 131 from air intake 111 and flow outsideof the tubular elements 410, but ultimately exit from the air output113. Although in this example embodiment, the vapor flow 530 is runninginside the tubular elements 410 and the air flow 520 runs outside of thetubular elements 410, one of ordinary skill in the art recognizes thatthere can be other arrangements for flow of fluids, e.g., one canconnect the vapor intake 112 and the vapor output 114 to the secondchannel and the air intake 111 and air output 113 to the first channel.

In one example embodiment, one or more tubular elements 410 can includea layer of carbon or carbon deposit. The carbon deposits can trap orabsorb dielectric molecules, thereby reducing the pressure within thetank. In one example embodiment, the carbon deposit can be on the insideof the one or more tubular elements 410. In another example embodiment,the carbon deposit can be on the outside of the one or more tubularelements 410.

In one example embodiment, the absorption unit 100 can include aplurality of modes of operation. In one mode of operation, themanagement system of the immersion cooling system detects an undesiredincrease in the pressure of the tank, e.g., a spike in the computingdemand of the computing device. The management system determines thatthe pressure has to be decreased. As such, the management system candirect vapor from the tank to the absorption unit (e.g., opening valvesconnecting the absorption unit 100 to the tank) to decrease the pressureof the tank. In this mode of operation of the absorption unit 100, thepump 121 can direct cool air (or ambient air) at the tubular elements410 so that the carbon deposit on these tubular elements 410 capturesthe excess vapor.

In one example, the carbon deposit 515 is on the inside of the tubularelements 410. In the mode of operation in which the absorption unit 100intends to capture the excess vapor (and thereby reduce the pressure ofthe tank), the valve 123 can connect the air filter 122 to the pump 121.The valve 123 can also connect the air output 113 to the exhaust pipe115. The management system can direct the excess vapor from the tank tothe vapor intake 112, and thus, the vapor can travel in one or more ofthe tubular elements 410 as the vapor flow 530. As the vapor flow 530travels in the tubular elements 410 (or even before or after the vaporflow 530 enters the tubular elements 410), the pump 121 can circulatethe air flow 520 to cool the tubular elements 410, and thus, facilitateabsorption of the vapor molecules by the carbon deposit 515. Forexample, the pump 121 can draw air from the air filter 122. The pump 121can cause the air to flow into the air intake 111 and travel on theoutside (i.e., in between) the tubular elements 410 as the air flow 520.When the air flow 520 travels in between the tubular elements 410, theair flow 520 can reduce the temperature of the tubular elements 410, andthus, accelerate absorption of the vapor molecules by the carbon deposit515. This system allows for cooling the carbon through convection and/orconduction. Once the air flow 520 exits the housing 131 via the airoutput 113, it can go to the valve 123, which connects the air output113 to the exhaust pipe 115. Thus, the air flow 520 can exit the systemand enter into the ambient air.

In one example, if the ambient air drawn from the air filter 122 iswarm, a cooler can chill the air before it enters the housing 131. Themanagement system can receive the temperature of the incoming air anddetermine whether any cooling is needed. For example, there can be oneor more temperature sensors located, e.g., at the air filter 122, beforeor after the pump 121, or before or after the air intake 111, which candetect the temperature of the incoming air and provide the temperatureto the management system. The management system can activate the coolerif reducing the temperature of the incoming air can improve vaporabsorption by the carbon deposit 515. For example, the management systemcan activate the cooler if the received temperature exceeds apredetermined threshold.

In one example embodiment, the management system can instruct the valve123 to cause recirculation of the air based on the temperature of theincoming air and the temperature of the air coming out of the air output113. For example, if the temperature of the air flowing out of the airoutput 113 is below the ambient temperature, it would be wasteful forthe absorption unit to send the air to the exhaust pipe 115 because theincoming air is warmer than the air coming out of the output 113. Themanagement system can receive the temperature of the incoming air andthe temperature of the air coming out of the air output 113 fromtemperature sensors located on the absorption unit 100, e.g., on the airfirst, pump, air intake 111, air output 113, etc.

In another mode of operation, the management system of the immersioncooling system detects an undesired decrease in the pressure of thetank, e.g., a drop in the computing demand of the computing device. Themanagement system determines that the pressure has to be increased. Assuch, the management system can direct vapor from the absorption unit tothe tank (e.g., opening valves connecting the absorption unit 100 to thetank) to increase the pressure of the tank. In this mode of operation ofthe absorption unit 100, the pump 121 can direct warm air at the tubularelements 410 so that the dielectric molecules captured by the carbondeposit on these tubular elements 410 vaporizes.

In one example, the carbon deposit 515 is on the inside of the tubularelements 410. In the mode of operation in which the absorption unit 100intends to vaporize captured dielectric fluid molecules (and therebyincrease the pressure of the tank), the valve 123 can connect the airoutput 113 to the pump 121. The valve 123 can disconnect the air filter122 and the exhaust pipe 115. The management system can direct theheater 125 to warm the air flow. The management system can also directthe pump 121 to circulate the air flow 520 to warm the tubular elements410, and thus, facilitate desorption of the vapor molecules by thecarbon deposit 515. For example, the pump 121 can cause the air to flowinto the air intake 111 and travel on the outside (i.e., in between) thetubular elements 410 as the air flow 520. When the air flow 520 travelsin between the tubular elements 410, the air flow 520 can increase thetemperature of the tubular elements 410, and thus, accelerate desorptionof the vapor molecules captured by the carbon deposit 515. Once the airflow 520 exits the housing 131 via the air output 113, it can go to thevalve 123, which connects the air output 113 to the pump 121 again.Thus, the air flow 520 can recirculate in the absorption unit andvaporize the dielectric fluid. The management system can direct thevaporized fluid (i.e., vapor flow 530) to the tank, and thus, the vaporcan increase the pressure of the tank.

In one example, if the ambient air is warm enough, the management systemcan direct the value 123 (and the actuator 124) to draw air from theambient environment surrounding the absorption unit 100 using the airfilter 122.

In one example embodiment, the air circulation system of the absorptionunit 100 can operate at atmospheric pressure. For example, theabsorption unit 100 can include one or more pressure sensors, e.g., atthe air filter 122, before or after the pump 121, before or after theair intake 111, before or after the air output 113, or at the valve 123.The pressure sensors can measure the pressure within the path of the airbeing circulated within the system. If the air pressure increases overatmospheric pressure (±a threshold tolerance), the management system candirect the valve 123 to release the air within the system. Specifically,the management system can receive a pressure reading from one or more ofthe pressure sensors. If the reading exceeds an allowable thresholdrange, the management system can direct the valve 123 to open theexhaust pipe 115 so that the system can be balanced again.

In one example embodiment, an immersion cooling system can include aplurality of absorption units 100. In one example, each absorption unitcan operate the same function (e.g., absorb or desorb vapor at the sametime). In another example, the absorption units can be complementary,e.g., one can be an absorption unit (some of the time or all the time)and the other one can be a desorption unit (some of the time or all thetime).

In one example embodiment, an immersion cooling system can useair-heated absorption units to maintain or control the pressure within atank. Air-heated absorption units can be more consistent fordistributing the heat within the housing of the absorption unit.Additionally, air-heated absorption units can last longer, and in casethey fail, it is easier to replace the heating element because theheating element is not located within the housing of the absorptionunit.

In one example embodiment, the absorption unit can include a bypassvalve for pressure release in emergencies. This is a safety feature ofthe absorption unit. The absorption unit can have a valve whichdisconnects it from the tank. Additionally or alternatively, theabsorption unit can have a bypass valve which can connect the vaporchannel of the absorption unit to the surrounding environment. Thebypass valve can be located, e.g., before the vapor intake 112 or afterthe vapor output 114. In the event there is an over-pressurizationcondition in the absorption unit, the valve can release the vapor to thesurrounding environment. Similarly, in the event there is anunder-pressurization condition in the absorption unit, the valve canreceive air from outside to normalize the condition within theabsorption unit. The management system can receive sensor data fromsensors placed at various locations in the absorption unit (e.g.,pressure or temperature sensors placed before or after the vapor intake112 or the vapor output 114). The data can indicate whether there is apressure condition within the tank. Based on the sensor data, themanagement system can order the bypass valve to open and/or normalizethe pressure.

In one example embodiment, in addition to pressure and temperature, themanagement system can receive and monitor data from the absorption unitrelating to electrical power of the absorption unit. The managementsystem ensures that current is running through the absorption unit. Themanagement system can shutdown the absorption unit if there is anovercurrent issue or if there is an over-pressurization condition.

In one example, the height and radius of the absorption unit can bedesigned based on the size of the vessel and the volume of fluid in thetank.

In one example embodiment, the management system can determine whetherthe air filter 122 needs to be replaced. For example, the absorptionunit 100 can include a pressure sensor before or after the air filter122. In the event the pressure reading of the pressure sensor deviatesfrom a normal reading by more than a predetermined tolerance range, themanagement system can determine that the air filter 122 needs to bereplaced. The management system can cause the immersion cooling systemto notify an operator of the system to replace the air filter. The airfilter 122 can filter dust and particulates.

Predictive Model

In one example embodiment, the management system can activate theabsorption unit based on a classification by a machine learning model ora predictive model. For example, the immersion cooling system canreceive the machine learning or predictive model from a central serveror the immersion cooling system can train the model based on data thatit collects over time. The predictive model can be developed by machinelearning algorithms. In an embodiment, the machine learning algorithmsemployed can include at least one selected from the group of gradientboosting machine, logistic regression, neural networks, and acombination thereof, however, it is understood that other machinelearning algorithms can be utilized. In an embodiment, the predictivemodel can be developed using foundational testing data generated by theimmersion cooling system and/or the central server. The foundationaltesting data can be stored in one or more databases.

The predictive model can include continuous learning capabilities. Insome examples, the database(s) can be continuously updated as new datais collected, e.g., each time the predictive model makes a prediction,the operator of the system can confirm or reject the model's predictionin a user interface of the immersion cooling system. The central servercan aggregate and provide the data to all the immersion cooling systemthat communicate with the central server. The new data can beincorporated into the training of the predictive model, so that thepredictive model reflects training based on data from various points intime and by various operators.

Initially, there may not be sufficient foundational testing dataavailable to develop the predictive model. Accordingly, the initialmodel development can be performed using predetermined classificationsas a proxy target and data available from other sources as features(e.g., data collected from other operators). By doing so, the predictivemodel can begin to form its understanding of conditions that requireintervention by the absorption unit. The results of this initialmodeling can support the initial status of the predictive model, and themodel can be continuously improved as data from operators becomesavailable. Once trained, the predictive model can be utilized to predictwhen the absorption unit manage a pressure condition for the tank.

In some examples, the predictive model can be stored on the immersioncooling system. Locally storing the model can realize the benefit ofreduced response times where predictions and trigger signals can be morequickly issued. In other examples, the predictive model can be stored onthe cloud or the central server, which can allow for centralizedmaintenance of the predictive model and greater accessibility of themodel for training. In examples where the predictive model is locallystored, the predictive model can be trained on the cloud andsynchronized across the local computing devices. Alternatively, thepredictive model can be trained continuously when locally stored andsynchronized across computing devices.

In one example embodiment, the training data for the predictive modelcan include temperature and pressure data. These data points can becollected by various sensors placed within and/or outside of theimmersion cooling system. For example, the sensors can be located withinthe absorption unit (e.g., to measure the pressure and/or temperaturewithin the unit), within the tank (e.g., to measure the pressure and/ortemperature within the tank), within the vessel and outside of thevessel. In one example embodiment, the training data can include anidentification of an event in connection with the immersion coolingsystem, e.g., startup, shutdown, hot swap, a spike or drop in theprocessing demand. In one example embodiment, the training data caninclude a type of compute, e.g., GPU processing, CPU processing orstorage of data. In one example embodiment, the management system canreceive data from other entities and make the prediction based on datareceived from these entities. For example, the management system canreceive data relating to the current weather, a weather forecast, a newsfeed (e.g., stock prices, cryptocurrency prices, market conditions andmetrics for cryptocurrencies, etc.).

In one example embodiment, the management system can feed various typesof data to the predictive model to determine whether the absorption unitneeds to be activated to address a predicted pressure condition. Forexample, the management system can provide the model with data such asthe current reading of the pressure and temperature sensors, a requestedevent, a current type of compute, a weather forecast, and a volatilitylevel in the cryptocurrency market. Based on these types of data, thepredictive model can predict whether the absorption unit needs to startaddressing a predicted future pressure condition. For example, whenthere is volatility in the cryptocurrency market, users place moreorders, and thus, the demand for processing compute increases. In suchconditions, the vapor pressure can increase in the tank, and thus, theabsorption unit will have to remove some of the excess vapor. Thepredictions by the prediction model can result in the absorption unitaddressing a pressure condition before the condition actually occurs,and thus, can reduce the response time for the absorption unit.

Process of Immersion Cooling

1. A method comprising:

at least partially submerging a computer component in a thermallyconductive, condensable dielectric fluid, wherein:

-   -   the computer component is mounted in a chassis comprising a        backplane for receiving power from a rack; and    -   the computer component is configured to dissipate heat in the        dielectric fluid when the computer component operates;

condensing, using a condenser, a gas phase of the dielectric fluid to aliquid phase of the dielectric fluid;

wherein the rack is within a tank comprising a pressure controller toreduce or increase an interior pressure of the tank.

2. The method of embodiment 1, wherein the tank has a computing powerdensity of at least 300 W of computing power dispersed over each squarefoot of space.

3. The method of embodiment 1, further comprising removing the chassisfrom the rack using a robot, wherein the robot is located within thetank.

4. The method of embodiment 3, further comprising delivering, using therobot, the chassis to an airlock, wherein the airlock is configured toallow access to an interior of the tank without significantly disruptinga pressure within the tank.

5. The method of embodiment 4, further comprising:

opening an inner door of the airlock;

placing the chassis in the airlock;

closing the inner door of the airlock;

equalizing a pressure of the airlock with an atmospheric pressure; and

opening the outer door of the airlock.

6. The method of embodiment 3, further comprising storing, using therobot, the chassis in a magazine.

7. The method of embodiment 6, wherein the magazine is located on aplatform including a supporting member, a rotating member and a rail.

8. The method of embodiment 3, wherein the robot is a gantry robotconfigured to remove, replace, or install the chassis.

9. The method of embodiment 8, wherein the gantry robot is configured tomove on a horizontal plane and drop down vertically.

10. The device of embodiment 9, wherein the robot is configured toremove, replace, or install a component of a power distribution system.

11. The method of embodiment 10, wherein the robot includes a grippingtool for grabbing the chassis.

12. The method of embodiment 1, wherein the tank is mounted within asuper structure which includes a plurality of tanks.

13. The method of embodiment 1, further comprising removing contaminantsfrom the dielectric fluid.

14. The method of embodiment 1, further comprising removing gaseouscontaminants.

15. The method of embodiment 1, further comprising providing power,network connection and process fluid to the tank.

16. The method of embodiment 1, wherein the tank comprises an opening onthe top and a removable lid.

17. The method of embodiment 1, wherein the tank comprises an interiorvolume of between about 100 cubic feet and about 300 cubic feet.

18. The method of embodiment 1, wherein the chassis does not include aheat sink and a fan.

19. The method of embodiment 1, wherein the chassis includes a bladeserver, a processor, a power supply or an interface card.

20. The device of embodiment 19, wherein the backplane is electricallyconnected to an interface card, which is a Cat6A or a Cat7 compatibleRJ45 interface for connection to a 1G or a 10G Ethernet interface.

Vessel Design and Configurations for Immersion Cooling

1. A device comprising:

a tank configured to hold thermally conductive, condensable dielectricfluid;

a pressure controller to reduce or increase an interior pressure of thetank;

a rack at least partially submerged within the dielectric fluid;

a condenser for condensing a gas phase of the dielectric fluid; and

a robot configured to move a chassis within the rack.

2. The device of embodiment 1, wherein the device comprises a modularskid comprising a plurality of forklift tubes.

3. The device of embodiment 1, wherein the tank has a computing powerdensity of at least 300 W of computing power dispersed over each squarefoot of space.

4. The device of embodiment 1, wherein an exterior of the deviceincludes a power input and a communication input.

5. The device of embodiment 4, wherein:

the power input and the communication input are electrically connectedto a box; and

the box, using a plurality of wires, distributes the power input and thecommunication input to the rack.

6. The device of embodiment 5, wherein the rack includes a backplanereceiver configured to distribute power and a communication signal tothe chassis.

7. The device of embodiment 6, wherein the chassis includes a backplaneconfigured to:

receive the power and the communication signal from the backplanereceiver of the rack; and

distribute the power and the communication signal to a computercomponent within the chassis.

8. The device of embodiment 5, wherein the plurality of wires do notinclude plastic insulation.

9. The device of embodiment 5, wherein the rack includes a transformer.

10. The device of embodiment 1, wherein the device is stackable.

11. The device of embodiment 1, wherein the device comprises a magazinefor storage of replacement components.

12. The device of embodiment 11, wherein the robot is configured toremove the chassis from the rack and place the chassis in the magazine.

13. The device of embodiment 12, wherein the magazine is located on aplatform which includes a rotating member, a supporting member and arail.

14. The device of embodiment 13, wherein the platform is configured toguide the magazine outside of the device.

15. The device of embodiment 1, wherein the device includes a desiccantconfigured to remove water vapor contamination from the device.

16. The device of embodiment 1, further comprising:

a sump area;

a pump; and

a filter;

wherein the pump is configured to remove the dielectric fluid from thesump area and pass the dielectric fluid through the filter beforedelivering the dielectric fluid to a bath portion of the tank.

17. The device of embodiment 1, wherein the dielectric fluid has aboiling point within a range of 20° C. to 100° C.

18. The device of embodiment 1, wherein the dielectric fluid comprises achemical with a formula of, (CF3)2CFCF2OCH3, C4F9OCH3, CF3CF2CF2CF2OCH3,hydrofluoro ethers or methoxy-nonaflurobutane.

19. The device of embodiment 1, further comprising a lock that precludesthe device from operating if any of a lid or a door of the device is notsecured.

20. The device of embodiment 1, further comprising a controllerconfigured to power down the device in the event of an unauthorizedaccess to the lid or the door.

Robotics and Automation for Immersion Cooling

1. A device comprising:

a tank configured to hold a thermally conductive, condensable dielectricfluid;

a pressure controller to reduce or increase an interior pressure of thetank;

a computer component at least partially submerged within the dielectricfluid;

a condenser for condensing a gas phase of the dielectric fluid; and

a robot configured to pick up the computer component.

2. The device of embodiment 1, further comprising an airlock.

3. The device of embodiment 2, wherein the airlock includes an innerdoor and an outer door.

4. The device of embodiment 3, wherein the airlock is configured toreceive an inert gas to purge the gas phase of the dielectric fluidbefore the outer door is opened.

5. The device of embodiment 3, wherein the robot is located outside thetank.

6. The device of embodiment 3, wherein the robot is located within thetank.

7. The device of embodiment 6, wherein the robot is configured to removethe computer component from a rack and deliver the computer component tothe airlock.

8. The device of embodiment 7, wherein the robot is further configuredto:

open the inner door of the airlock;

place the computer component in the airlock;

close the inner door of the airlock;

equalize a pressure of the airlock with an atmospheric pressure; and

open the outer door of the airlock.

9. The device of embodiment 8, further comprising a second robot locatedoutside of the tank.

10. The device of embodiment 9, wherein the second robot is configuredto remove the computer component from the airlock when the outer door isopen.

11. The device of embodiment 9, wherein the second robot is configuredto place the computer component within a storage slot.

12. The device of embodiment 9, wherein the airlock is configured toequalize the pressure of the airlock after the outer door is closed.

13. The device of embodiment 1, wherein the device is configured toreceive instructions from a server located outside of the device.

14. The device of embodiment 1, wherein the computer component islocated within a chassis showing an asset tag.

15. The device of embodiment 14, wherein the robot is configured to scanthe asset tag and relay the asset tag to a management system.

16. The device of embodiment 1, wherein the robot is a gantry robotconfigured to remove, replace, or install the computer component.

17. The device of embodiment 16, wherein the gantry robot is configuredto move horizontally and vertically.

18. The device of embodiment 1, wherein the robot is configured toremove, replace, or install a component of a power distribution system.

19. The device of embodiment 18, wherein the component of the powerdistribution system is a transformer or a power supply.

20. The device of embodiment 1, wherein the robot includes a grippingtool for gripping the computer component.

Ballast Blocks for Immersion Cooling

1. A device comprising:

a tank comprising:

-   -   a bath portion for holding thermally conductive, condensable        dielectric fluid and a computer component; and    -   a shelf portion configured to hold at least one ballast block;

a pressure controller to reduce or increase an interior pressure of thetank;

a condenser for condensing a gas phase of the dielectric fluid; and

a robot configured to pick up the computer component.

2. The device of embodiment 1, wherein a bottom point of the bathportion has a lower height than a height of the shelf portion.

3. The device of embodiment 1, wherein the bath portion is configuredfor the computer component to be at least partially submerged in thedielectric fluid.

4. The device of embodiment 3, wherein the computer component is a bladeserver, a processor, a power supply, or a transformer.

5. The device of embodiment 1, wherein a level of the dielectric fluidis high enough to cover at least part of the shelf portion.

6. The device of embodiment 1, wherein the shelf portion is next to thecondenser.

7. The device of embodiment 6, wherein the shelf portion is configuredto receive condensed dielectric fluid from the condenser.

8. The device of embodiment 1, wherein the ballast block is configuredto occupy a volume of the tank above the shelf to displace thedielectric fluid from the shelf to an area over the bath portion.

9. The device of embodiment 1, wherein the ballast block includes aplurality of riser feet for allowing the dielectric fluid to flowunderneath the ballast block.

10. The device of embodiment 1, wherein the ballast block is not solublein the dielectric fluid.

11. The device of embodiment 1, wherein the ballast block is made from ametal, a rubber, a silicone, or a polymer.

12. The device of embodiment 1, wherein the ballast block is denser thanthe dielectric fluid.

13. The device of embodiment 1, wherein the ballast block has a handle,cut out or a plate for removal or replacement of the ballast block.

14. The device of embodiment 13, wherein the robot is configured to liftthe ballast block using the handle, the cut out or the plate.

15. The device of embodiment 1, wherein the ballast block is configuredto interlock with another ballast block from a top side or a bottom sideof the ballast block.

16. The device of embodiment 15, wherein the interlocking prevents theother ballast block from sliding.

17. The device of embodiment 15, wherein the other ballast block isconfigured to be on the top side or the bottom side of the ballastblock.

18. The device of embodiment 15, wherein the ballast block comprisesrecessed portions on the top side of the ballast block such that a riserfeet of the other ballast block is configured to lock in one of therecessed portions of the ballast block.

19. The device of embodiment 1, wherein the ballast block is configuredto span at least 40% of an entire length of the shelf portion.

20. The device of embodiment 1, wherein the ballast block has an outerdimensions of about 2 feet long, about 8 inches wide and about 1 inchtall.

Server Case for Immersion Cooling

1. A device comprising:

a tank configured to hold thermally conductive, condensable dielectricfluid;

a pressure controller to reduce or increase an interior pressure of thetank;

a chassis at least partially submerged within the dielectric fluid;

a condenser for condensing a gas phase of the dielectric fluid; and

a robot configured to pick up the chassis.

2. The device of embodiment 1, wherein the chassis does not require aheat sink or a fan.

3. The device of embodiment 1, wherein the chassis includes a bladeserver.

4. The device of embodiment 1, wherein the chassis includes a processor,a power supply or an interface card.

5. The device of embodiment 4, wherein the interface card is a Cat6A ora Cat7 compatible RJ45 interface for connection to a 1G or a 10GEthernet interface.

6. The device of embodiment 1, wherein the chassis is removably attachedto a rack.

7. The device of embodiment 6, wherein the chassis includes a backplaneto provide a slot-in interface between the chassis and the rack.

8. The device of embodiment 7, wherein the backplane is configured todistribute power and signals received from the rack within the chassis.

9. The device of embodiment 8, wherein the backplane is configured totransmit power and data to a blade server via a cable.

10. The device of embodiment 1, wherein the chassis is a substantiallyrectangular box comprising a back wall and two sidewalls, wherein theback wall has a plurality of holes to facilitate circulation of thedielectric fluid within the chassis.

11. The device of embodiment 10, wherein the chassis comprises a guiderail on each of the two sidewalls.

12. The device of embodiment 1, wherein the chassis comprises a mountinginterface for holding computer components.

13. The device of embodiment 1, wherein the chassis comprises a plateand the robot is configured to lift the chassis using the plate.

14. The device of embodiment 1, wherein the chassis includes amicrocontroller.

15. The device of embodiment 14, wherein the microcontroller isconfigured to:

receive sensor data from a sensor mounted on the chassis, the sensordata indicating whether the chassis is properly placed in a rack; and

transmit the sensor data to a management system.

16. The device of embodiment 14, wherein the microcontroller isconfigured to:

receive a power signal from a management system; and

transmit the power signal to a switch configured to cutoff the powerwithin the chassis.

17. The device of embodiment 14, wherein the microcontroller isconfigured to:

receive operation data from a computer component mounted within thechassis; and

transmit the operation data to the management system.

18. The device of embodiment 14, wherein the microcontroller isconfigured to control electrical and communication facilities of a bladeserver.

19. The device of embodiment 1, wherein the chassis comprises an RFIDtag.

20. The device of embodiment 19, wherein the robot is configured to scanthe RFID tag and transmit a signal to a management system.

Vapor Management for Immersion Cooling Using Bellows

1. A device comprising:

a tank configured to hold thermally conductive, condensable dielectricfluid and a computer component;

a pressure controller to reduce or increase an interior pressure of thetank;

a vapor management system for condensing a gas phase of the dielectricfluid; and

a robot configured to pick up the computer component.

2. The device of embodiment 1, wherein the vapor management systemincludes a condensing structure within the tank.

3. The device of embodiment 2, wherein the condensing structure includesa thermally conductive tube, a coil or radiator fins.

4. The device of embodiment 2, wherein the condensing structure isconfigured to be coupled to a source of cooling liquid so that thecooling liquid passes through the condensing structure.

5. The device of embodiment 2, wherein the device is configured to chillthe cooling liquid using evaporative cooling or dry cooling towers.

6. The device of embodiment 2, wherein the vapor management systemincludes an incoming pipe and an outgoing pipe.

7. The device of embodiment 6, wherein the incoming pipe is configuredto receive cooling liquid from a chilled cooling liquid source and guidethe cooling liquid through the condensing structure.

8. The device of embodiment 6, wherein the outgoing pipe is configuredto receive cooling liquid from the condensing structure and return thecooling liquid to the chilled cooling liquid source.

9. The device of embodiment 1, wherein the vapor management systemincludes a storage unit for storage of the dielectric fluid.

10. The device of embodiment 9, wherein the vapor management system isconfigured to direct the dielectric fluid in the tank from the storageunit.

11. The device of embodiment 1, wherein the vapor management systemincludes a vapor storage unit for storage of vapor of the dielectricfluid.

12. The device of embodiment 11, wherein the vapor storage unit is abellows.

13. The device of embodiment 12, wherein the bellows are configured toinflate or deflate to maintain the interior pressure of the tank.

14. The device of embodiment 12, wherein the bellows comprises one ormore pouches.

15. The device of embodiment 11, wherein the vapor storage unitcomprises a valve for allowing air into the vapor management system toreduce a temperature of the vapor of the dielectric fluid.

16. The device of embodiment 15, wherein the vapor storage unit isoperably connected to a carbon bed to separate the vapor of thedielectric fluid from air.

17. The device of embodiment 16, wherein the carbon bed comprises adesorption heater configured to heat the carbon bed to raise thetemperature of the carbon bed.

18. The device of embodiment 1, wherein the vapor management systemcomprises a filter.

19. The device of embodiment 17, wherein the filter is configured toremove air and water vapor.

20. The device of embodiment 1, wherein the vapor management system:

comprises an inert gas storage unit; and

is configured to introduce an inert gas from the inert gas storage unitinto the tank during a startup operation or a shutdown operation.

What is claimed is:
 1. A system comprising: a tank, wherein the tank isconfigured to hold a liquid phase and a gas phase of a fluid; astructure within the tank configured to hold one or more computercomponents to be at least partially submerged within the liquid phase ofthe fluid during an operation of the system; an absorption unitincluding a carbon element; and a controller configured to regulate theabsorption unit, and wherein the absorption unit includes: a housing; anair intake; an air output; a vapor intake; and a vapor output.
 2. Thesystem of claim 1, wherein: the housing includes a plurality of tubes;the vapor intake is coupled to the vapor output using the plurality oftubes; and each tube is configured to pass incoming vapor from the vaporintake to the vapor output.
 3. The system of claim 2, wherein the airintake is coupled to the air output using the housing.
 4. The system ofclaim 3, wherein the vapor intake, the vapor output, the air intake, theair output and the plurality of tubes are configured such that thehousing includes two separate channels for guiding fluids through thehousing: a first channel for the incoming vapor, wherein the vaporintake is configured to provide the incoming vapor through the pluralityof tubes to the vapor output; and a second channel for the incoming air,wherein the air intake is configured flow the incoming air outside ofthe plurality of tubes to the air output.
 5. The system of claim 4,wherein an inside of each of the plurality of tubes is coated with alayer of carbon.
 6. The system of claim 5, wherein the layer of carbonis configured to absorb the incoming vapor.
 7. The system of claim 6,wherein the layer of carbon is configured to desorb vapor in response toreceiving heat.
 8. The system of claim 1, further comprising an airpump, a heater and a 4-way valve.
 9. The system of claim 8, wherein the4-way valve is connected to: an air filter; an exhaust pipe; the airpump; and the air output.
 10. The system of claim 9, wherein the airpump is connected to the air intake through the heater.
 11. The systemof claim 10, wherein the air pump is configured to draw an incoming airfrom the air filter through the 4-way valve and pass the incoming airthrough the heater and to the air intake.
 12. The system of claim 11,wherein the controller is configured to regulate the heater to raise thetemperature of the housing.
 13. The system of claim 10, wherein thecontroller is configured to operate the 4-way valve such that the airpump draws an incoming air from the air filter or the air output. 14.The system of claim 10, wherein the controller is configured to operatethe 4-way valve such that air is released from the air output throughthe exhaust pipe in the event of an emergency.
 15. The system of claim14, further comprising a pressure sensor.
 16. The system of claim 15,wherein the controller is configured to detect the emergency if thepressure sensor indicates an internal pressure within the housing thatexceeds a threshold.
 17. The system of claim 1, further comprising avapor emergency release.
 18. The system of claim 17, wherein the vaporemergency release includes a valve configured to release an incomingvapor in the event of an emergency.
 19. The system of claim 1, whereinthe controller is configured to determine to activate the air pump basedon a prediction by a prediction model.